Recording medium, reporduction apparatus, recording method, integrated circuit, program and reporduction method

ABSTRACT

AVClip recorded in BD-ROM is obtained by multiplexing a graphics stream and a video stream. The graphics stream is a PES packet sequence that includes 1) PES packets storing graphics data (ODS) and 2) PES packets storing control information (PCS). In each ODS, values of DTS and PTS indicate a timing of decoding start for corresponding graphics data, and a timing of decoding end for corresponding graphics data, respectively. In each PCS, a value of PTS indicates a display timing of corresponding decoded graphics data combined with the video stream.

TECHNICAL FIELD

The present invention relates to a recording medium such as BD-ROM, andto a reproduction apparatus. The present invention particularly relatesto a technology for realizing subtitle display and interactive display,by means of graphics.

BACKGROUND ART

Subtitle display by means of graphics has an important mission ofconveying words uttered by the characters in a work, to people in everyarea of the world. One conventional technology for realizing subtitledisplay is the subtitle application of ETSI EN 300 743 standard (ETSI:European Telecommunication Standards Institute). The subtitleapplication is a video stream to be reproduced together with subtitledisplay by means of graphics. Here, the graphics that correspond tosubtitle are displayed as a data stream of MPEG2 standard. The datastream is a sequence of PES packets, where each PES packet has a PTS(presentation time stamp). ETSI EN 300 743 standard defines the timingof subtitle display in a subtitle application. This standard establishessynchronization between a moving picture and graphics, in which graphicsis displayed when corresponding images in a video stream are displayed.

When subtitle application is to be provided for BD-ROM, there is a needfor further enhancing the resolution level of the graphics. To be morespecific, the resolution level is desired to be improved to the level of1920*1080. However, realization of such a high definition incurs anenormous amount of decoding load at reproduction.

The ETSI EN 300 743 standard defines reproduction control for performingdecoding at the time designated by the PTS, and for displaying itstraight away. When this is applied, an enormous amount of decoding loadwill be concentrated in the reproduction apparatus at points immediatelybefore display. Such concentration of load forces the hardware/softwarecompetence of the reproduction apparatus to be high, so as to realizegraphics display. If such a condition becomes essential for reproductionapparatuses, production cost for reproduction apparatuses will riseremarkably, which will prevent such reproduction apparatuses from beingcommonly used.

DISCLOSURE OF THE INVENTION

The object of the present invention is to provide a recording mediumthat realizes high definition level of graphics display, as well asavoiding rise in production cost.

So as to achieve the above-described object, the present inventionprovides a recording medium storing therein a digital stream that isobtained by multiplexing a graphics stream and a video stream, thegraphics stream being a packet sequence that includes a data packetstoring graphics data and a control packet storing control information,wherein

the data packet has a time stamp whose value indicates a decoding timeof the graphics data, and

the control packet has a time stamp whose value indicates a time atwhich the graphics data, after being decoded, is displayed combined withthe video stream.

The period in which graphics is decoded is indicated by the time stampof the packet storing the graphics, and display of the graphics isdefined by the value of the time stamp assigned to corresponding controlinformation. Therefore in the present invention, “state of alreadydecoded but not yet displayed”, in other words, a state in whichdecompressed graphics is buffered, is defined on the reproductiontimeline.

By defining such a buffering period, it becomes possible to avoidconcentration of an enormous amount of decoding load to one point. Inaddition, if the use of the hardware resource for decodingsimultaneously contends other processing, buffering period can beprovided so as to relocate the graphics decoding period, therebyavoiding such contention.

Here, if this buffering concept is introduced so as to achieve theaforementioned object, technicians pursuing the development of such areproduction apparatus will be at a loss about the extent of the memoryto be mounted, so as to guarantee normal operation. Meanwhile,technicians who produce subtitle application will also be anxious aboutwhether the subtitle application of their own can be assuredlyreproduced by the reproduction apparatus. All these are because thememory occupancy for this buffering will change chronologically, in thecourse of reproduction on the reproduction timeline. If thechronological change in memory occupancy stays unknown, the anxieties ofthese technicians will not be wiped away.

So as to solve this problem, it is desirable to have a construction inwhich the control information includes type information that indicates amemory management start, the time stamp of the control packet is apresentation time stamp, and the control packet further includes adecode time stamp whose value indicates a point of a reproductiontimeline of the digital stream, which corresponds to the memorymanagement start, and a time at which the control information is read toa memory.

According to this construction, a memory management start is indicatedby a decode time stamp of a packet storing control information.Therefore, by referring to the decode time stamp, it becomes possible toknow on which point of the reproduction timeline each buffer for thedecoder model should be flashed. If the flash point is considered as astarting point of memory management, it is easy to grasp thechronological occupancy transition of the buffer that stores controlinformation, the buffer that stores graphics before being decoded, andthe buffer that stores graphics after being decoded. By changing thevalue of this decode time stamp, it is possible to adjust thechronological transition of the state of the buffers. According to suchadjustment, it becomes possible to avoid buffer overflow at thereproduction apparatus. Therefore, it becomes easy to implementhardware/software at the development stage of reproduction apparatus.

In addition, since it becomes easy to grasp and adjust the chronologicaltransition, verification becomes accordingly easy as to whether thegraphics stream obtained by authoring satisfies the restraints of thedecoder model that the BD-ROM standard assumes. Therefore a person incharge of authoring can proceed with his authoring operation, under acondition where the graphics of his creation is assured to be operatednormally.

In assuming the decoder model of BD-ROM, a further constituting elementwill become necessary so as to carry out the present invention. In thisdecoder model of BD-ROM, the decoder main body (i.e. processor) ofgraphics is independent from the controller main body (i.e. controller)for updating the graphics. The reason why the decoder main body isprovided independently from the updating controller main body is toperform advanced updating such as displaying and deleting of graphicsgradually, which is useful for a case when the graphics is a subtitle,for example. When the updating controller main body is an independentbody from the decoder main body, processor-controller connection willneed to be closer. This is because, after the processor completesdecoding of the graphics data, the controller has to perform updatewithout delay.

The manner in which the decoding completion of the processor is notifiedto the controller depends on the manner in which the processor and thecontroller are implemented in the apparatus. If the processor and thecontroller are implemented as programs, notification will be performedby intra-process communication. If the processor and the controller areimplemented as independent hardware components from each other, thennotification will be performed by an interrupting signal. The amount oftime lag of such notification also depends on the implementation mannerin the apparatus. If the implementation necessitates a large time lag ofnotification, there will be a case where updating of graphics cannot besynchronized with the display rate of the moving picture.

So as to prevent such a case from occurring, it is desirable to have aconstruction in which the value of the presentation time stamp isobtained by adding a predetermined value to the value of the decode timestamp, where the predetermined value is based on: a longer one of aperiod required for clearing of a screen, and a period required fordecoding of the graphics data; and a period required for writing of thegraphics data to the screen.

The presentation time stamp of a packet storing graphics indicates adecoding ending time, and the presentation time stamp of a packetstoring control information indicates a time obtained by adding apredetermined period to the decoding ending time. Therefore only byreferring to the presentation time stamps, the controller can performupdating at an adequate timing without receiving from the processor anydecoding-completion notification of graphics data. If such update isperformed, it becomes possible to assure update synchronized with thedisplay rate of the moving picture, regardless of the manner ofimplementation in the reproduction apparatus.

Since closer processor-controller connection is realized regardless ofthe manner of processor-controller implementation in the reproductionapparatus, it becomes possible to maintain a degree of flexibility inthe apparatus designing, as well as facilitating manufacturingapparatuses at low cost.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of use of a recording medium according tothe present invention.

FIG. 2 illustrates a structure of a BD-ROM.

FIG. 3 is a diagram schematically illustrating a structure of an AVClip.

FIG. 4A illustrates a structure of a presentation graphics stream.

FIG. 4B illustrates a PES packet obtained after functional segments areconverted.

FIG. 5 illustrates a logical structure that is made of various kinds offunctional segments.

FIG. 6 illustrates a relation between a display position of a subtitleand an Epoch.

FIG. 7A illustrates syntax to define a Graphics Object in an ObjectDefinition Segment (ODS).

FIG. 7B illustrates syntax of a Palette Definition Segment (PDS).

FIG. 8A illustrates syntax of a Window Definition Segment (WDS).

FIG. 8B illustrates syntax of a Presentation Composition Segment (PCS).

FIG. 9 illustrates an example of a description of a Display Set forsubtitling.

FIG. 10 illustrates an example of a description of the WDS and PCS in aDS1.

FIG. 11 illustrates an example of a description of the PCS in a DS2.

FIG. 12 illustrates an example of a description of the PCS in a DS3.

FIG. 13 is an example of a description of a Display Set when Cut-In/Outis performed, illustrating along a timeline.

FIG. 14 is an example of a description of a Display Set when Fade-In/Outis performed, illustrating along a timeline.

FIG. 15 is an example of a description of a Display Set when Scrollingis performed, illustrating along a timeline.

FIG. 16 is an example of a description of a Display Set when Wipe-In/Outis performed, illustrating along a timeline.

FIG. 17 is a diagram comparing two cases: a window has four GraphicsObjects, and a window has two Graphics Objects.

FIG. 18 illustrates an example of an algorithm for calculating a decodeduration.

FIG. 19 is a flowchart of the algorithm of FIG. 18.

FIGS. 20A and B are flowcharts of the algorithm of FIG. 18.

FIG. 21A illustrates a case in which each window has an ObjectDefinition Segment.

FIGS. 21B and C are timing charts showing orders among numerals referredto in FIG. 18.

FIG. 22A illustrates a case in which each window has two ObjectDefinition Segments.

FIGS. 22B and C are timing charts showing orders among numerals referredto in FIG. 18.

FIG. 23A describes a case in which each of two Windows includes an ODS.

FIG. 23B illustrates a case in which a decode period (2) is longer thana total of a clearing period (1) and a write period (31).

FIG. 23C illustrates a case in which a total of the clearing period (1)and the write period (31) is longer than the decode period (2).

FIG. 24 illustrates chronological transition of update described in anexample in the present specification.

FIG. 25A illustrates four Display Sets that are described so as toperform the above explained update.

FIG. 25B is a timing chart showing settings of DTS and PTS of functionalsegments included in the four Display Sets.

FIG. 26 illustrates an internal structure of a reproduction apparatusaccording to the present invention.

FIG. 27 illustrates sizes of write rates Rx, Rc, and Rd, Graphics Plane8, Coded Data Buffer 13, Object Buffer 15, and Composition Buffer 16.

FIG. 28 is a timing chart illustrating a pipeline processing by thereproduction apparatus.

FIG. 29 illustrates a timing chart in a pipeline processing of a case inwhich decoding of the ODS ends before clearing of the Graphics Plane iscompleted.

FIG. 30 is a timing chart showing chronological transition of the amountof accumulation at the Graphics Plane 8.

FIG. 31 is a flowchart illustrating a process of a loading operation ofa functional segment.

FIG. 32 shows an example of multiplexing.

FIG. 33 illustrates a manner in which a DS10 is loaded to the Coded DataBuffer 13 of the reproduction apparatus.

FIG. 34 illustrates a case where a normal reproduction is performed.

FIG. 35 illustrates loading of a DS1, the DS10, and a DS20 in a normalreproduction as performed in FIG. 34.

FIG. 36 is a flowchart showing a process performed by the Graphicscontroller 17.

FIG. 37 is a flowchart showing a process performed by the GraphicsController 17.

FIG. 38 a flowchart showing a process performed by the GraphicsController 17.

FIG. 39 illustrates a pipeline process of the reproduction apparatusbased on the PTS of the PDS.

FIG. 40 is a diagram describes a significance of the END in the pipelineprocess of the reproduction apparatus.

FIG. 41 is a diagram schematically illustrates a structure of AVClipaccording to the second embodiment.

FIG. 42A and FIG. 42B are diagrams about an interactive screen accordingto the second embodiment.

FIG. 43 illustrates a data structure of Interactive Composition Segment.

FIG. 44 shows a relationship between an ODS included in a DSn, and ICS.

FIG. 45 shows a screen composition at the display timing of arbitrarypicture data “pt1”.

FIG. 46 shows an example of setting for button information in ICS.

FIG. 47 illustrates a state transition of button A-button D.

FIG. 48 illustrates images of ODS11, 21, 31, and 41, as one example.

FIG. 49 illustrates images of ODS11-19 for the button A, as one example.

FIG. 50 illustrates the button-state groups and the order of ODS inDisplay Set.

FIG. 51 illustrates a state transition of an interactive screen in whichthe button-state groups of FIG. 50 are disposed.

FIG. 52 illustrates the order of ODS in Display Set.

FIG. 53 shows difference in arrangement of ODS in S-ODSs, between a caseof default_selected_button_number=0, and a case ofdefault_selected_button_number=button B.

FIGS. 54A and 54B shows a value of ΣSIXE(DSn[ICS.BUTTON[i]]), in a casewhen N-ODSs include plural ODS constituting the buttons A-D, and S-ODSsinclude plural ODS constituting the buttons A-D.

FIG. 55 shows a timing of synchronized display by means of ICS.

FIG. 56 shows how DTS and PTS are set in a case where an initial displayof an interactive screen is constituted by plural ODSs, and where thedefault_selected_button is valid.

FIG. 57 shows how DTS and PTS are set in a case where an initial displayof an interactive screen is constituted by plural ODSs, and where thedefault_selected_button is invalid.

FIG. 58 illustrates the content of an Object Buffer 15 in comparisonwith a Graphics Plane 8.

FIG. 59 illustrates an operation performed by the Graphics controller 17at the initial display time.

FIG. 60 illustrates an operation performed by the Graphics controller 17when the interactive screen update is performed according to1stUserAction(MoveRight).

FIG. 61 illustrates an operation performed by the Graphics controller 17when the interactive screen update is performed according to the1stUserAction(MoveDown).

FIG. 62 illustrates an operation performed by the Graphics controller 17when the interactive screen update is performed according to the1stUserAction(Activated).

FIG. 63 is a timing chart illustrating a pipeline processing performedby the reproduction apparatus.

FIG. 64 is a timing chart illustrating a pipeline processing by thereproduction apparatus in a case where the default selected buttonchanges dynamically.

FIG. 65 is a timing chart illustrating chronological transitions inoccupancy of the Graphics Plane 8, the Object Buffer 15, the Coded Databuffer 13, and the Composition buffer 16.

FIG. 66 is a flowchart showing a process of loading operation ofSegment.

FIG. 67 shows an example of multiplexing.

FIG. 68 illustrates a manner in which a DS10 is loaded to the Coded DataBuffer 13 of the reproduction apparatus.

FIG. 69 illustrates a case where a normal reproduction is performed.

FIG. 70 illustrates loading of a DS1, the DS10, and a DS20 in a normalreproduction as performed in FIG. 69.

FIG. 71 is a flowchart illustrating a main routine of the processingperformed by the Graphics controller 17.

FIG. 72 is a flowchart illustrating a processing for realizing synccontrol that uses time stamp.

FIG. 73 is a flowchart showing a process of an operation to write to theGraphics Plane 8.

FIG. 74 is a flowchart illustrating an automatic activation processingfor the default selected button.

FIG. 75 is a flowchart showing an animation display process.

FIG. 76 is a flowchart illustrating a process of UO operation.

FIG. 77 is a flowchart illustrating a process of current button changingoperation.

FIG. 78 is a flowchart illustrating a process of numerical valueinputting operation.

FIG. 79 illustrates a method of manufacturing a BD-ROM that records thePCS explained in the first embodiment.

FIG. 80 illustrates a method of manufacturing a BD-ROM that records thePCS explained in the second embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment

A First Embodiment of a recording medium according to the presentinvention is explained below.

FIG. 1 illustrates an example of use of the recording medium. In thedrawing, BD-ROM 100 is the recording medium according to the presentinvention. The BD-ROM 100 is used for providing data of movie works to aHome Theatre System structured by a reproduction apparatus 200, atelevision 300, and a remote controller 400.

The recording medium according to the present invention is manufacturedby an improvement in an application layer of a BD-ROM. FIG. 2illustrates a structure of the BD-ROM.

In the drawing, the BD-ROM is shown at a bottom of the drawing, and atrack on the BD-ROM is shown above the BD-ROM. The track is actually ina spiral shape on the disk, but shown in a line in the drawing. Thetrack includes a lead-in area, a volume area, and a lead-out area. Thevolume area in this drawing has a physical layer, a file system layer,and an application layer. At a top of the drawing, an application formatof the BD-ROM is illustrated using a directory structure. As illustratedin the drawing, the BD-ROM has a directory BDMV under the rootdirectory, and the BDMV directory contains a file for storing an AVClipwith an extension M2TS (XXX.M2TS), a file for storing administrativeinfo for the AVClip with an extension CLPI (XXX.CLPI), and a file fordefining a logical PlayList (PL) for the AVClip with an extension MPLS(YYY.MPLS) By forming the above application format, it is possible tomanufacture the recording medium according to the present invention. Ina case in which there are more than one file for each kind, it ispreferable to provide three directories named STREAM, CLIPINF, andPLAYLIST under the BDMV to store the files with the same extension inone directory. Specifically, it is desirable to store the files with theextension M2TS in the STREAM, the files with the extension CLPI in theCLIPINF, and the files with the extension MPLS in the PLAYLIST.

An explanation about the AVClip (XXX.M2TS) in the above applicationformat is given below.

The AVClip (XXX.M2TS) is a digital stream in MPEG-TS format (TS isTransport Stream) obtained by multiplexing a video stream, at least oneaudio stream, and a presentation graphics stream. The video streamrepresents pictures of the film, the audio stream represents sound ofthe film, and the presentation graphics stream represents subtitles ofthe film. FIG. 3 is a diagram schematically illustrating a structure ofthe AVClip.

The AVClip (XXX.M2TS) is structured in a following manner. The videostream made of plural vide frames (picture pj1, pj2, and pj3), and theaudio stream made of plural audio frames (top row of the drawing) arerespectively converted into a line of PES packets (second row of thedrawing), and then into a line of TS packets (third row of the drawing).The presentation graphics stream (bottom row of the drawing) isconverted into a line of PES packets (second to bottom row of thedrawing), and then into a line of TS packets (third to bottom row of thedrawing). The three lines of TS packets are multiplexed, and thus theAVClip (XXX.M2TS) is constituted.

In the drawing, only one presentation graphics stream is multiplexed.However, in a case in which the BD-ROM is compatible to plurallanguages, a presentation graphics stream for each language ismultiplexed to constitute the AVClip. The AVClip constituted in theabove manner is divided into more than one extent, like ordinarycomputer files, and stored in areas in the BD-ROM.

Next, the presentation graphics stream is explained. FIG. 4A illustratesa structure of the presentation graphics stream. A top row indicates theTS packet line to be multiplexed to the AVClip. A second to the top rowindicates the PES packet line that constitutes a graphics stream. ThePES packet line is structured by retrieving payloads out of TS packetshaving a predetermined PID, and connecting the retrieved payloads.

A third to the top row indicates the structure of the graphics stream.The graphics stream is made of functional segments named a PresentationComposition Segment (PCS), a Window Definition Segment (WDS), a PaletteDefinition Segment (PDS), an Object Definition Segment (ODS), and an ENDof Display Set Segment (END). Among the above functional segments, thePCS is called a screen composition segment, and the WDS, PDS, ODS, andEND are called definition segments. The PES packet and each of thefunctional segments correspond one to one, or one to plurality. In otherwords, one functional segment is either recorded in the BD-ROM afterconverted into one PES packet, or after divided into fragments andconverted into more than one PES packet.

FIG. 4B illustrates the PES packet obtained by converting the functionalsegments. As shown in the drawing, the PES packet is made of a packetheader and the payload, and the payload is a substantial body of afunctional segment. The packet header includes a DTS and a PTScorresponding to the functional segment. The DTS and PTS included in thepacket header are hereinafter referred to as the DTS and PTS of thefunctional segment.

The above described various kind of functional segments constitute alogical structure as illustrated in FIG. 5. FIG. 5 illustrates thelogical structure that is made of the various kinds of functionalsegments. In the drawing, a top row illustrates Epochs, a middle rowillustrates Display Sets (DS), and a bottom row illustrates thefunctional segments.

Each of the DS shown in the middle row is a group of functional segmentsthat compose graphics for one screen, among all of the plural functionalsegments that constitute the graphics stream. Broken lines in thedrawing indicate the DS to which the functional segments in the bottomrow belong, and show that a series of the functional segments of thePCS, WDS, PDS, ODS, and END constitute one DS. The reproductionapparatus is able to generate graphics for one screen by reading thefunctional segments that constitute the DS.

The Epochs shown in the top row indicate time periods, and memorymanagement is consecutive time wise along a timeline of the AVClipreproduction in one Epoch. One Epoch also represents a group of datathat is assigned to the same period of time. The memory referred to hereare the Graphics Plane that stores the graphics for one screen, and anObject Buffer that stores decompressed graphics data. Theconsecutiveness of the memory management means a flash of the GraphicsPlane or of the Object Buffer does not occur in the Epoch, and erasingand rendering of the graphics are only performed in a predeterminedrectangular area on the Graphics Plane (the flash here indicatesclearing of all contents of the stored data in a plane or a buffer). Asize and a position of the rectangular area are fixed during one Epoch.As long as the erasing and rendering of the graphics are only performedin the predetermined rectangular area on the Graphics Plane, a syncreproduction between the picture and the graphics is guaranteed. Inother words, the Epoch is a unit in the reproducing timeline, and inthis unit, the picture and the graphics are guaranteed to be reproducedsynchronously. When moving the area, in which the graphics are erasedand rendered, to a different position, it is necessary to define a pointon the timeline to move the area, and a period after the point becomes anew Epoch. The sync reproduction is not guaranteed at a boarder betweentwo Epochs.

In viewing an actual film, one Epoch is a time period in which subtitlesare displayed in the same rectangular area on the screen. FIG. 6illustrates a relation between the position of the subtitles and theEpochs. In an example illustrated by the drawing, the positions at whichthe five subtitles “Actually . . . ”, “I was hiding”, “my feelings.”, “Ialways”, and “loved you.” are shown move according to the picture in thefilm. Specifically, the subtitles “Actually . . . ”, “I was hiding”, and“my feelings.” appear at the bottom of the screen, while the subtitles“I always” and “loved you.” are shown at the top of the screen. Theposition of the rectangular area moves in order that the subtitles areout of the way of pictures when viewing the screen, consideringvisibility of the film. A time period during which the subtitles appearat the bottom is an Epoch 1, and a subsequent time period during whichthe subtitles appear at the top is an Epoch 2. The Epochs 1 and 2 eachhave a different area in which the subtitles are rendered. The area inthe Epoch 1 is a Window 1 positioned at the bottom of the screen, andthe area in the Epoch 2 is a Window 2 positioned at the top of thescreen. The memory management is consecutive in each of the Epochs 1 and2, and accordingly, rendering of the subtitles in the Windows 1 and 2 issynchronous with the pictures.

Next, details about the Display Set (DS) are described.

Broken lines hkl1 and hkl2 in FIG. 5 indicate which functional segmentat the middle row belongs to which Epoch. A series of DS “Epoch Start”,“Acquisition Point”, and “Normal Case” constitute the Epoch at the topraw. The “Epoch Start”, “Acquisition Point”, “Normal Case”, and “EpochContinue” are types of the DS, and an order between the “AcquisitionPoint” and “Normal Case” does not matter and either of them may comefirst.

The Epoch Start is a DS that has a display effect of “new display”,which indicates a start of a new Epoch. Because of this, the Epoch Startcontains all functional segments needed to display a new composition ofthe screen. The Epoch Start is provided at a position which is a targetof a skip operation of the AVClip, such as a chapter in a film.

The Acquisition Point is a DS that has a display effect of “displayrefresh”, and is identical in content used for rendering graphics withthe Epoch Start which is a preceding DS. The Acquisition Point is notprovided at a starting point of the Epoch, but contains all functionalsegments needed to display the new composition of the screen. Therefore,it is possible to display the graphics without fail when a skipoperation to the Acquisition Point is performed. Accordingly, with theAcquisition Point, it is possible to compose a screen in the middle ofthe Epoch.

The Acquisition Point is provided at a position that could be a targetfor the skip operation. An example of such a position is a position thatcould be specified when performing a time search. The time search is anoperation in response to a user's input of a time to start reproducingfrom a reproducing point corresponding to the time specified by theuser. The time is specified roughly, by 10 minutes or by 10 seconds forexample, and accordingly, points at which the reproduction starts areprovided at a 10 minute interval, or a 10 second interval, for example.By providing the Acquisition Point at the points at which thereproduction may start, it is possible to perform reproduction smoothlyafter the time search.

The Normal Case is a DS that has a display effect of “display update”,and contains only elements that are different from the precedingcomposition of the screen. Specifically, when subtitles in a DSv is thesame as subtitles in a DSu but the screen is displayed differently inthe DSv and DSu, the DSv is provided so as to include only the PCS andmakes the DSv the Normal Case. By this, it becomes unnecessary toprovide an ODS with the same content as the content of the ODS in thepreceding DS, and a data size in the BD-ROM may be reduced. On the otherhand, because the DS as the Normal Case contains only the difference, itis not possible to compose the screen using the Normal Case alone.

The Epoch Continue indicates that Epoch continues across a boundary ofAVClips. If Composition State of one DSn is set as Epoch Continue, ifthe DSn exists on an AVClip different from that of the DSn-1 positionedimmediately before the DSn, the DSn and DSn-1 will belong to a sameEpoch. Therefore even if AVClip branching occurs between these two DS,there will be no graphics plane/object buffer flash.

Details of the Definition Segments (ODS, WDS, and PDS) are explainedbelow. The Object Definition Segment (ODS) is a functional segment thatdefines the Graphics Object. An explanation of the Graphics Object isgiven first. A selling point of the AVClip recorded in the BD-ROM is itsresolution as high as hi-vision, and therefore the resolution for theGraphics Object is set at 1920×1080 pixels. Because of the highresolution of 1920×1080 pixels, it is possible to display a specificcharacter style for the subtitles clearly on the screen. As for colorsof the subtitles, a bit length of an index value for each pixel (ColorDifference Red Cr, Color Difference Blue Cb, Luminance Y, andTransparency T) is 8 bits, and thus it is possible to chose any 256colors out of full color (16,777,216 colors) for the subtitles. Thesubtitles realized by the Graphics Object are rendered by positioningtexts on a transparent background.

Syntax of ODS to define the Graphics Object is shown in FIG. 7A. The ODSis made of segment_type indicating that the segment is the ODS,segment_length indicating a data length of the ODS, object_id uniquelyidentifying the Graphics Object corresponding to the ODS in the Epoch,object_version_number indicating a version of the ODS within the Epoch,last_in_sequence_flag, and object_data_fragment which is a consecutivesequence of bytes corresponding to a part or all of Graphics Object.

The object_id is for uniquely identifying the Graphics Objectcorresponding to the ODS in the Epoch. The Epoch of the graphics streamcontains more than one ODS having the same ID. The ODS having the sameID also have the same width and height, and are assigned with a commonarea in the Object Buffer. After one of the ODS having the same ID isread in the common area, the read ODS is overwritten by a subsequent ODShaving the same ID. By overwriting the ODS that is read to the ObjectBuffer by the subsequent ODS having the same ID as the reproduction ofthe vide stream proceeds, the graphics by the ODS is updatedaccordingly. A size constraint that the width and height of the GraphicsObject having the same ID should be the same is applied only during oneEpoch, and the Graphics Objects in different Epochs may have differentsizes.

Explanations about last_in_sequence_flag and object_data_fragment aregiven next. In some cases, it is not possible to store the decompressedgraphics that constitutes the subtitle in one ODS due to a payloadconstraint of the PES packet. In such cases, the graphics is split intoa series of consecutive fragments, and one fragment is set to theobject_data_fragment. When one Graphics Object is stored as more thanone fragment, every fragment except a last fragment has the same size.The last fragment is less than or equal to the size of previousfragments. The ODS carrying the fragments appear in the same sequentialorder in the DS, with an end of the sequence indicated by the ODS havingthe last_in_sequence_flag. Although the above described syntax of theODS is based on a premise that the fragments are stacked in from thepreceding PES, the fragments may be stacked so that each PES contains ablank part.

Next, the Palette Definition Segment (PDS) is explained. The PDS is usedto define a palette for a color conversion. FIG. 7B shows syntax of thePDS. The PDS is made of segment_type indicating that the segment is thePDS, segment_length indicating a data length of the PDS, palette_iduniquely identifying the palette contained in the PDS,palette_version_number indicating a version of the PDS within the Epoch,and palette_entry_id specifying an entry number of the palette. Thepalette_entry_id indicates the Color Difference Red (Cr_value), theColor Difference Blue (Cb_value), Luminance (Y_value), and Transparency(T_value).

Next, an explanation about the Window Definition Segment (WDS) is givenbelow.

The WDS is used to define the rectangular area on the Graphics Plane. Asdescribed in the above, the memory management is sequential only whenerasing and rendering is performed within a certain area on the GraphicsPlane. The area on the Graphics Plane is defined by the WDS and called“Window”. FIG. 8A illustrates syntax of the WDS. As shown by thedrawing, the WDS is made of segment_type indicating that the segment isthe WDS, segment_length indicating a data length of the WDS, window_iduniquely identifying the Window on the Graphics Plane,window_horizontal_position specifying a horizontal address of a top leftpixel of the Window on the Graphics Plane, window_vertical_positionspecifying a vertical address of the top left pixel of the Window on theGraphics Plane, window_width specifying a width of the Window on theGraphics Plane, and window_height specifying a height of the Window onthe Graphics Plane.

Ranges of values that the window_horizontal_position,window_vertical_position, window_width, and window_height may take areexplained below. A coordinate system for those values is within an areaon the Graphics Plane, and whose size is indicated two-dimensionally bythe window_height for a height and the window_width for a width.

The window_horizontal_position specifies the horizontal address of thetop left pixel of the Window on the Graphics Plane, and is within arange of 0 to (window_width)−1. Also, the window_vertical_positionspecifies the vertical address of the top left pixel of the Window onthe Graphics Plane, and is within a range of 0 to (window_height)−1.

The window_width specifies the width of the Window on the GraphicsPlane. The specified width falls within a range of 1 to(video_width)−(window_horizontal_position). Further, the window_heightspecifies the height of the Window on the Graphics Plane, and thespecified height is within a range of 1 to(video_height)−(window_vertical_position).

The position and size of the Window on the Graphics Plane for each Epochare defined by the window_horizontal_position, window_vertical_position,window_width, and window_height. Accordingly, it is possible to adjustthe position and size of the Window at authoring, so that the Window inone Epoch appears at the position that does not come in the way of thepicture when viewing the film. By this, the visibility of the subtitlesbecomes higher. Because the WDS is defined for each Epoch, it ispossible to adjust the position of the Window according to the picture,even if the picture changes in the course of time. As a result, thequality of the film is maintained as high as in a case where thesubtitles are incorporated in the main body of the film.

Next, the End of Display Set Segment (END) is explained. The ENDprovides an indication that a transmission of the DS is completed. TheEnd is inserted into a stream immediately after a last ODS in one DS.The End is made of segment_type indicating that the segment is the ENDand segment_length indicating a data length of the END. The END does notinclude any other element that requires a further explanation.

Next, an explanation about the Presentation Composition Segment (PCS) isgiven below.

The PCS is a functional segment that is used for composing aninteractive display. FIG. 8B illustrate syntax of the PCS. As shown inthe drawing, the PCS is made of segment_type, segment_length,composition_number, composition_state, palette_update_flag, palette_id,and composition_object 1-m.

The composition_number identifies the Graphics Update in the DS byvalues in a range of 0 to 15. If the Graphics Update exists between thehead of the Epoch and the PCS, the composition_number is incrementedevery time the Graphics Update occurs.

The composition_state indicates the type of the DS in which the PCS iscontained, Normal Case, Acquisition Point, or Epoch Start.

The palette_update_flag indicates that the PCS describes a Palette onlyDisplay Update. The Palette only Display Update indicates that only thepalette is updated from an immediately previous palette. Thepalette_update_flag field is set to “1”, if the Palette only. DisplayUpdate is performed.

The palette_id identifies the palette to be used in the Palette onlyDisplay Update.

The composition_object 1-m indicate how to control each Window in the DSto which the PCS belong. A broken line wd1 in FIG. 8B is to detail aninternal syntax for composition_object i. The composition_object i ismade of object_id, window_id, object_cropped_flag,object_horizontal_position, an object_vertical_position, andcropping_rectangle information 1-n.

The object_id identifies the ODS in a Window corresponding to thecomposition_object i.

The window ididentifies the Window to which the Graphics Object isallocated in the PCS. Up to two Graphics Objects may be assigned to oneWindow.

The object_cropped_flag is used to switch between display and no-displayof a cropped Graphics Object in the Object Buffer. When theobject_cropped_flag is set to “1”, the cropped Graphics Object isdisplayed in the Object Buffer, and if set to “0”, the Graphics Objectis not displayed.

The object_horizontal_position specifies a horizontal address of a topleft pixel of the Graphics Object in the Graphics Plane.

The object_vertical_position specifies a vertical address of the topleft pixel of the Graphics Object in the Graphics Plane.

The cropping_rectangle information 1-n are elements used when theobject_cropped_flag is set to “1”. A broken line wd2 is to detail aninternal syntax for cropping_rectangle information i. As shown by thebroken line wd2, the cropping_rectangle information i is made of fourfields, object_cropping_horizontal_position,object_cropping_vertical_position, object_cropping_width, andobject_cropping_height.

The object_cropping_horizontal_position specifies a horizontal addressof a top left corner of a cropping rectangle to be used during renderingof the Graphics Object in the Graphics Plane. The cropping rectangle isa cropping frame that is used to specify and crop a part of the GraphicsObject, and corresponds to Region in the ETSI EN 300 743 standard.

The object_cropping_vertical_position specifies a vertical address ofthe top left corner of the cropping rectangle to be used duringrendering of the Graphics Object in the Graphics Plane.

The object_cropping_width specifies a width of the cropping rectangle.

The object_cropping_height specifies a height of the cropping rectangle.

A specific example of the PCS is detailed below. In the example, thesubtitles “Actually . . . ”, “I was hiding”, and “my feelings.” as shownin FIG. 6 appear gradually by writing to the Graphics Plane 3 times asthe picture proceeds. FIG. 9 is an example of description for realizingsuch a subtitle display. An Epoch in the drawing includes a DS1(EpochStart), a DS2 (Normal Case), and a DS3 (Normal Case). The DS1contains a WDS for specifying the Window in which the subtitles aredisplayed, an ODS for specifying the line “Actually . . . I was hidingmy feelings.”, and a first PCS. The DS2 contains a second PCS, and theDS3 contains a third PCS.

FIGS. 10-12 illustrate examples of the WDS and PCS contained in the DS.FIG. 10 shows an example of the PCS in the DS1.

In FIG. 10, the window_horizontal_position and thewindow_vertical_position of the WDS are indicated by a LP1, a positionof the top left pixel of the Window on the Graphics Plane. Thewindow_width and window_height indicate the width and height of theWindow, respectively.

In FIG. 10, the object_cropping_horizontal_position andobject_cropping_vertical_position indicate a reference point ST1 of thecropping rectangle in the coordinate system in which an origin is thetop left pixel of the Graphics Object. The cropping rectangle is an areahaving the width from the ST to the object_cropping_width, and theheight from the ST to the object_cropping_height (a rectangle shown by aheavy-line frame). The cropped Graphics Object is positioned within arectangle shown by a broken-line frame cp1, with a reference point inthe coordinate system with an origin at the object_horizontal_positionand object_vertical_position (the top left pixel of the Graphics Object)in the Graphics Plane. By this, the subtitle “Actually . . . ” iswritten to the Window on the Graphics Plane, and then combined with themovie picture and displayed on the screen.

FIG. 11 shows an example of the PCS in the DS2. The WDS in the DS2 isnot explained, because the WDS in the DS2 is the same as the WDS in theDS1. A description of the cropping information in the DS2 is differentfrom the description of the cropping information shown in FIG. 10.

In FIG. 11, the object_cropping_horizontal_position andobject_cropping_vertical_position in the cropping information indicate atop left pixel of the subtitle “I was hiding” out of “Actually . . . Iwas hiding my feelings.” in the Object Buffer. The object_cropping_widthand object_cropping_height indicates a width and a height of a rectanglecontaining the subtitle “I was hiding”. By this, the subtitle “I washiding” is written to the Window on the Graphics Plane, and thencombined with the movie picture and displayed on the screen.

FIG. 12 shows an example of the PCS in the DS3. The WDS in the DS3 isnot explained, because the WDS in the DS3 is the same as the WDS in theDS1. A description of the cropping information in the DS3 is differentfrom the description of the cropping information shown in FIG. 10.

In FIG. 12, the object_cropping_horizontal_position andobject_cropping_vertical_position in the cropping information indicate atop left pixel of the subtitle “my feelings.” out of “Actually . . . Iwas hiding my feelings.” in the Object Buffer. The object_cropping_widthand object_cropping_height indicates a width and a height of a rectanglecontaining the subtitle “my feelings.”. By this, the subtitle “myfeelings.” is written to the Window on the Graphics Plane, and thencombined with the movie picture and displayed on the screen.

By describing the DS1, DS2, and DS3 as explained above, it is possibleto achieve an effect of displaying the subtitles on the screen. It isalso possible to achieve other kinds of effect, and descriptionprotocols for realizing other effects are explained below.

First, a description protocol for a Cut-In/Out effect is explained. FIG.13 shows an example of the description of the DS when Cut-In/Out isperformed, illustrating along a timeline.

In the drawing, x and y in Window (x,y,u,v) respectively indicate valuesof the window_vertical_position and window_horizontal_position, and uand v respectively indicate values of the window_width andwindow_height. Also in the drawing, a and b in Cropping Rectangle(a,b,c,d) respectively indicate values of theobject_cropping_vertical_position andobject_cropping_horizontal_position, and c and d indicate values of theobject_cropping_width and object_cropping_height, respectively. DisplaySets DS11, DS12, and DS13 are at points t11, t12, and t13 on thereproduction timeline in the drawing.

The DS11 at the point t11 includes a PCS#0 in which thecomposition_state is “Epoch Start” and the object_cropped_flag is “0”(no_cropping_rectangle_visible), a WDS#0 having a statement for a Windowin a width 700×height 500 at (100,100) in the Graphics Plane, a PDS#0,an ODS#0 indicating a subtitle “Credits:”, and an END.

The DS12 at the point t12 includes a PCS#1 whose composition_state is“Normal Case” and indicating a crop operation of the Graphics Object tobe in a 600×400 size from (0,0) in the Object Buffer(cropping_rectangle#0 (0,0,600,400)), and positioning the croppedGraphics Object at the coordinates (0,0) in the Graphics Plane (onWindow#0 (0,0)).

The DS13 at the point t13 includes a PCS#2 whose composition_state is“Normal Case” and in which the object_cropped_flag is set to “0” so asto erase the cropped Graphics Object (no_cropping_rectangle_visible).

With the above explained Display Sets, the subtitle “Credits:” isno-display at the t11, appears at the t12, then becomes no-display atthe t13 again, and the Cut-In/Cut-Out effect is realized.

Secondly, a description protocol for a Fade-In/Out effect is explained.FIG. 14 shows an example of the description of the DS when Fade-In/Outis performed, illustrating along a timeline. Display Sets DS21, DS22,DS23, and DS24 are at points t21, t22, t23, and t24 on the reproductiontimeline in the drawing.

The DS21 at the point t21 includes a PCS#0 whose composition_state is“Epoch Start” and indicating the crop operation of the Graphics Objectto be in a 600×400 size from (0,0) in the Object Buffer(cropping_rectangle#0 (0,0,600,400)), and positioning the croppedGraphics Object at the coordinates (0,0) in the Graphics Plane (onWindow#0 (0,0)), a WDS#0 having a statement for a Window in a width700×height 500 at (100,100) in the Graphics Plane, a PDS#0, an ODS#0indicating a subtitle “Fin”, and an END.

The DS22 at the point t22 includes a PCS#1 whose composition_state is“Normal Case”, and a PDS#L. The PDS#1 indicates the same level of Cr andCb as the PDS#0, but a luminance indicated by the PDS#L is higher thanthe luminance in the PDS#0.

The DS23 at the point t23 includes a PCS#2 whose composition_state is“Normal Case”, a PDS#2, and an END. The PDS#2 indicates the same levelof Cr and Cb as the PDS#L, but the luminance indicated by the PDS#2 islower than the luminance in the PDS#1.

The DS24 at the point t24 includes a PCS whose composition_state is“Normal Case” and the object_cropped_flag is “0”(no_cropping_rectangle_visible), and an END.

Each DS specifies a different PDS from a preceding DS, and accordingly,the luminance of the Graphics Object that is rendered with more than onePCS in one Epoch becomes gradually high, or low. By this, it is possibleto realize the effect of Fade-In/Out.

Next, a description protocol for a Scrolling is explained. FIG. 15 showsan example of the description of the DS when Scrolling is performed,illustrating along a timeline. Display Sets DS31, DS32, DS33, and DS34are at points t31, t32, t33, and t34 on the reproduction timeline in thedrawing.

The DS31 at the point t31 includes a PCS#0 whose composition_state isset to “Epoch Start” and object_cropped_flag is “0”(no_cropping_rectangle_visible), a WDS#0 having a statement for a Windowin a width 700×height 500 at (100,100) in the Graphics Plane, a PDS#0,an ODS#0 indicating a subtitle “Credits: Company”, and an END.

The DS32 at the point t32 includes a PCS#1 whose composition_state is“Normal Case” and indicating the crop operation of the Graphics Objectto be in a 600×400 size from (0,0) in the Object Buffer(cropping_rectangle#0 (0,0,600,400)), and positioning the croppedGraphics Object at the coordinates (0,0) in the Graphics Plane (onWindow#0 (0,0)). An area of the 600×400 size from (0,0) in the ObjectBuffer includes apart “Credits:” of the subtitle “Credits: Company”shown in two lines, and thus the part “Credits:” appears on the GraphicsPlane.

The DS33 at the point t33 includes a PCS#2 whose composition_state is“Normal Case” and indicating the crop operation of the Graphics Objectto be in a 600×400 size from (0,100) in the Object Buffer(cropping_rectangle#0 (0,100,600,400)), and positioning the croppedGraphics Object at the coordinates (0,0) in the Graphics Plane (onWindow#0 (0,0)). The area of the 600×400 size from (0,100) in the ObjectBuffer includes the part “Credits:” and a part “Company” of the subtitle“Credits: Company” shown in two lines, and thus the parts “Credits:” and“Company” appear in two lines on the Graphics Plane.

The DS34 at the point t34 includes a PCS#3 whose composition_state is“Normal Case” and indicating the crop operation of the Graphics Objectto be in a 600×400 size from (0,200) in the Object Buffer(cropping_rectangle#0 (0,200,600,400)), and positioning the croppedGraphics Object at the coordinates (0,0) in the Graphics Plane (onWindow#0 (0,0)). The area of the 600×400 size from (0,200) in the ObjectBuffer includes the part “Company” of the subtitle “Credits: Company”shown in two lines, and thus the part “Company” appears on the GraphicsPlane. By the above PCS description, it is possible to scroll down thesubtitle in two lines.

Finally, a description protocol for a Wipe-In/Out effect is explained.FIG. 16 shows an example of the description of the DS when Wipe-In/Outis performed, illustrating along a timeline. Display Sets DS21, DS22,DS23, and DS24 are at points t21, t22, t23, and t24 on the reproductiontimeline in the drawing.

The DS51 at the point t51 includes a PCS#0 whose composition_state isset to “Epoch Start” and the object_cropped_flag is “0”(no_cropping_rectangle_visible), a WDS#0 having a statement for a Windowin a width 700×height 500 at (100,100) in the Graphics Plane, a PDS#0,an ODS#0 indicating a subtitle “Fin”, and an END.

The DS52 at the point t52 includes a PCS#1 whose composition_state is“Normal Case” and indicating the crop operation of the Graphics Objectto be in a 600×400 size from (0,0) in the Object Buffer(cropping_rectangle#0 (0,0,600,400)), and positioning the croppedGraphics Object at the coordinates (0,0) in the Graphics Plane (onWindow#0 (0,0)). An area of the 600×400 size from (0,0) in the ObjectBuffer includes the subtitle “Fin”, and thus the subtitle “Fin” appearson the Graphics Plane.

The DS53 at the point t53 includes a PCS#2 whose composition_state is“Normal Case” and indicating the crop operation of the Graphics Objectto be in a 400×400 size from (200,0) in the Object Buffer(cropping_rectangle#0 (200,0,400,400)), and positioning the croppedGraphics Object at the coordinates (200,0) in the Graphics Plane (onWindow#0 (200,0)). By this, an area indicated by coordinates (200,0) and(400,400) in the Window becomes a display area, and an area indicated bycoordinates (0,0) and (199,400) becomes a no-display area.

The DS54 at the point t54 includes a PCS#3 whose composition_state is“Normal Case” and indicating the crop operation of the Graphics Objectto be in a 200×400 size from (400,0) in the Object Buffer(cropping_rectangle#0 (400,0,200,400)), and positioning the croppedGraphics Object at the coordinates (400,0) in the Graphics Plane (onWindow#0 (400,0)). By this, an area indicated by coordinates (0,0) and(399,400) becomes the no-display area.

By this, as the no-display area becomes larger, the display area becomessmaller, and thus the Wipe-In/Out effect is realized.

As described above, various effects such as Cut-In/Out, Fade-In/Out,Wipe-In/Out, and Scrolling may be realized using corresponding scripts,and therefore it is possible to make various arrangements in renderingthe subtitles.

Constraints for realizing the above effects are as follows. In order torealize the Scrolling effect, operations for clearing and redrawing ofthe Window becomes necessary. Taking the example of FIG. 15, it isnecessary to perform “window clear” to erase the Graphics Object“Credits:” at the t32 from the Graphics Plane, and then to perform“window redraw” to write a lower part of “Credits:” and an upper part of“Company” to the Graphics Plane during an interval between the t32 andt33. Given that the interval is the same as an interval of video frames,a transfer rate between the Object Buffer and the Graphics Planedesirable for the Scrolling effect becomes an important point.

Here, a constraint about how large the Window may be is looked into. AnRc is the transfer rate between the Object Buffer and the GraphicsPlane. A worst scenario here is to perform both of the Window clear andWindow redraw at the rate Rc. In this case, each of the Window clear andWindow redraw is required to be performed at a rate half of Rc (Rc/2).

In order to make the Window clear and Window redraw synchronized with avideo frame, an equation below is need to be satisfied.Window size×Frame Rate □ Rc/2

If the Frame Rate is 29.97, Rc is expressed by an equation below.Rc=Window size×2×29.97

In rendering the subtitles, the Window size accounts for at least 25% to33% of the Graphics Plane. A total number of pixels in the GraphicsPlane is 1920×1080. Taking that an index bit length per pixel is 8 bits,a total capacity of the Graphics Plane is 2 Mbytes (01920×1080×8).

Taking that the Window size is ¼ of the total capacity of the GraphicsPlane, the Window size becomes 500 Kbytes (=2 Mbytes/4). By substitutingthis value to the above equation, Rc is calculated to be 256 Mbps (=500Kbytes×2×29.97). If the rate for the Window clear and Window redraw maybe a half or a quarter of the frame rate, it is possible to double orquadruple the size of the Window even if the Rc is the same.

By keeping the Window size 25% to 33% of the Graphics Plane anddisplaying the subtitles at the transfer rate of 256 Mbps, it ispossible to maintain the sync display between the graphics and the moviepicture, no matter what kind of display effect is to be realized.

Next, the position, size, and area of the Window are explained. Asexplained above, the position and area of the Window does not change inone Epoch. The position and the size of the Window set to be the sameduring one Epoch because it is necessary to change a target writeaddress of the Graphics Plane if the position and the size change, andchanging the address causes an overhead that lowers the transfer ratefrom the Object Buffer to the Graphics Plane.

A number of Graphics Objects per Window has a limitation. The limitationof the number is provided in order to reduce the overhead intransferring decoded Graphics Object. The overhead here is generatedwhen setting the address of an edge of the Graphics Object, and the morea number of edges, the more the overhead is generated.

FIG. 17 shows examples in comparison, an example in which a Window hasfour Graphics Objects and another example in which a Window has twoGraphics Objects. The number of the edges of the example with fourGraphics Objects is twofold of the number of the edges of the examplewith two Graphics Objects.

Without the limitation in the number of the Graphics Object, it becomesunknown how many overheads could be generated in transferring theGraphics, and thus the load for the transfer increases and decreasesdrastically. On the other hand, when a maximum number of the GraphicsObject in a Window is two, the transfer rate may be set taking up to 4overhead into account. Accordingly, it is easier to set the number of aminimum transfer rate.

Next, an explanation about how the DS having the PCS and ODS is assignedto the timeline of the AVClip. The Epoch is a period of time in which amemory management is consecutive along the reproduction timeline. Sincethe Epoch is made of more than one DS, how to assign the DS to thereproduction timeline of the AVClip is important. The reproductiontimeline of the AVClip is a timeline for specifying timings for decodingand reproducing of each piece of picture data that constitute the videostream multiplexed to the AVClip. The decoding and reproducing timingson the reproduction timeline are expressed at an accuracy of 90 KHz. ADTS and PTS that are attached to the PCS and ODS in the DS indicatetimings for a synchronic control on the reproduction timeline. Theassigning of the Display Set to the reproduction timeline meansperforming the synchronic control using the DTS and PTS attached to thePCS and ODS.

First, how the synchronic control is performed using the DTS and PTSattached to the ODS is explained below.

The DTS indicates, at the accuracy of 90 KHz, a time when the decodingof the ODS starts, and the PTS indicates a time when the decoding ends.

The decoding of the ODS does not finish at once, and has a certainlength of time. In response to a request for clearly indicating astarting point and an ending point of a decode duration, the DTS and PTSof the ODS respectively indicate the times when the decoding starts andends.

The value of the PTS indicates the deadline, and therefore it isnecessary that the decoding of the ODS has to be completed by the timeindicated by the PTS and the decompressed Graphics Object is written tothe Object Buffer on the reproduction apparatus.

The decode starting time of any ODSj in a DSn is indicated by a DTS(DSn[ODS]) at the accuracy of 90 KHz. Adding a maximum length of thedecode duration to the DTS(DSn[ODS]) is the time when the decoding ofthe ODSj ends.

When a size of the ODSj is “SIZE(DSn[ODSj])” and a decoding rate of theODS is an “Rd”, the maximum time required for decoding indicated bysecond is expressed in “SIZE(DSn[ODSj])//Rd”. The symbol “//” indicatesan operator for a division with rounding up after a decimal place.

By converting the maximum time period into a number expressed at theaccuracy of 90 KHz and adding to the DTS of the ODSj, the time when thedecoding ends (90 KHz) indicated by the PTS is calculated.

The PTS of the ODSj in the DSn is expressed in a following equation.PTS(DSn[ODSj])=DTS(DSn[ODSj])+90,000×(SIZE(DSn[ODSj])//Rd)

Further, it is necessary that a relation between two succeeding ODS,ODSj and ODSj+1, satisfies a following equation.PTS(DSn[ODSj])≦DTS(DSn[ODSj+1])

Next, settings of the DTS and PTS of the PCS are explained.

It is necessary that the PCS is loaded to the Object Buffer on thereproduction apparatus before the decode starting time (DTS(DSn[ODS1]))of a first ODS (ODS1) in the DSn, and before the time (PTS (DSn [PDS1]))when a first PDS (PDS1) in the DSn becomes valid. Accordingly, it isnecessary that the DTS is set so as to satisfy following equations.DTS(DSn[PCS])≦DTS(DSn[ODS1])DTS(DSn[PCS])≦PTS(DSn[PDS1])

Further, the PTS of the PCS in the DSn is expressed in a followingequation.PTS(DSn[PCS])≧DTS(DSn[PCS])+decodeduration(DSn)

The “decodeduration (DSn)” indicates a time duration for decoding allthe Graphics Objects used for updating PCS. The decode duration is not afixed value, but does not vary according to a status of the reproductionapparatus and a device or a software mounted to the reproductionapparatus. When the Object used for composing a screen of a DSn.PCSn isa DSn.PCSn.OBJ[j], the decodeduration(DSn) is affected by time (i)needed for clearing the Window, decode durations (ii) for decoding aDSn.PCSn.OBJ, and time (iii) needed for writing of the DSn.PCSn.OBJ.When the Rd and Rc are set, the decode_duration (DSn) is always thesame. Therefore, the PTS is calculated by calculating lengths of thesedurations in authoring.

The calculation of the decode_duration is performed based on a programshown in FIG. 18. FIGS. 19, 20A and 20B are flowcharts schematicallyshowing algorithms of the program. An explanation about the calculationof the decode_duration is given below referring to these drawings. Inthe flowchart shown in FIG. 19, first, a PLANEINITIALZE function iscalled (Step S1 in FIG. 19). The PLANEINITIALZE function is used forcalling a function for calculating a time period necessary to initializethe Graphics Plane for rendering the DS. In the Step S1 in FIG. 19, thefunction is called with arguments DSn, DSn.PCS.OBJ[0], anddecode_duration.

The following explains the PLANEINITIALZE function in reference to FIG.20A. In the drawing, initialize_duration is a variable indicating areturn value of the PLANEINITIALZE function.

Step S2 in FIG. 20 is an if statement for switching operations dependingon whether or not the page_state in the PCS in the DSn indicates theEpoch Start. If the page_state indicates the Epoch Start(DSn.PCS.page_state==epoch_start, Step S2=Yes in FIG. 18), a time periodnecessary to clear the Graphics Plane is set to an initialize_duration(Step S3).

When the transfer rate Rc between the Object Buffer and the GraphicsPlane is 256,000,000 as described in the above, and the total size ofthe Graphics Plane is set to video_width*video_height, the time periodnecessary to clear is “video_width*video_height//256,000,000”. Whenmultiplied by 90.000 Hz so as to express at the time accuracy of PTS,the time period necessary to clear the Graphics Plane is“90,000×video_width*video_height//256,000,000”. This time period isadded to the initialize_duration.

If the page_state does not indicate the Epoch Start (Step S2=No), a timeperiod necessary to clear Window[i] defined by the WDS is added to theinitialize_duration for all Windows (Step S4). When the transfer rate Rcbetween the Object Buffer and the Graphics Plane is 256,000,000 asdescribed in the above and a total size of Winodow[i] that belongs tothe WDS is ΣSIZE(WDS.WIN[i]), the time period necessary to clear is“ΣSIZE(WDS.WIN[i])//256,000,000”. When multiplied by 90.000 Hz so as toexpress at the time accuracy of PTS, the time period necessary to clearthe Windows that belong to the WDS is“90,000×ΣSIZE(WDS.WIN[i])//256,000,000”. This time period is added tothe initialize_duration, and the initialize_duration as a result isreturned. The above is the PLANEINITIALZE function.

Step S5 in FIG. 19 for switching operations depending on whether thenumber of the Graphics Objects in the DSn is 2 or 1(if(DSn.PCS.num_of_object==2, if(DSn.PCS.num_of_object==1 in FIG. 18),and if the number is 1 (Step S5), a waiting time for decoding theGraphics Object is added to the decode_duration (Step S6). Calculationof the waiting time is performed by calling a WAIT function(decode_duration+=WAIT(DSn, DS.PCS.OBJ[0], decode_duration) in FIG. 18).The function is called using arguments set to DSn, DSn.PCS.OBJ[0],decode_duration, and a return value is wait_duration.

FIG. 20B is a flowchart showing an operation of the WAIT function.

In the flowchart, the decode_duration of an invoker is set as acurrent_duration. An object_definition_ready_time is a variable set tothe PTS of the Graphics Object of the DS.

A current_time is a variable set to a total value of thecurrent_duration and the DTS of the PCS in the DSn. When theobject_definition_ready_time is larger than the current_time (Yes toStep S7, if(current_time<object_definition_ready_time)), thewait_duration as the return value is set to be a difference between theobject_definition_ready_time and the current_time (Step S8,wait_duration+=object_definition_ready_time−current_time). Thedecode_duration is set to the time period that the return value of theWAIT function added to the time period necessary for re-drawing theWindow, (90,000*(SIZE(DSn.WDS.WIN[0]))//256,000,000).

The above explanation is for the case in which the number of theGraphics Object is one. In Step S5 in FIG. 5, it is judged if the numberof the Graphics Objects is two. If the number of the Graphics Objects inthe DSn is more than two (if(DSn.PCS.num_of_object==2) in FIG. 18), theWAIT function is called using OBJ[0] in the PCS as an argument, and adda return value to the decode_duration (Step S10).

In a succeeding Step S11, it is judged if the Window to which the OBJ[0]of the DSn belongs is the same as the Window to which the GraphicsObject [1] belongs (if(DSn.OBJ[0,] window_id==DSn.PCS.OBJ[1].window_id).If the Window is the same, the WAIT function is called using OBJ[1] asan argument, and add a return value wait_duration to the decode_duration(Step S12), and add the time necessary to redraw the Window to whichOBJ[0] belong (90,000*(SIZE (DSn.WDS.OBJ[0]. window_id))//256,000,000)to the decode_duration (Step S13).

If it is judged that the Windows are different (Step S11, “different”),the time necessary to redraw the Window is added to which OBJ[0]belong(90,000*(SIZE (DSn.WDS.OBJ[0]. window_id))//256,000,000) to thedecode_duration (Step S15), the WAIT function is called using OBJ[1] asan argument, and add a return value wait_duration to the decode_duration(Step S16), and the time necessary to redraw the Window to which OBJ[1]belong (90,000*(SIZE(DSn.WDS.OBJ[0]. window_id))//256,000,000) to thedecode_duration (Step S17).

The decode_duration is calculated by the above algorithm. A specificmanner in which the PTS of the OCS is set is explained below.

FIG. 21A illustrates a case in which one ODS is included in one Window.FIGS. 21B and 21C are timing charts showing values in an order of timethat are referred to in FIG. 18. A bottom line “ODS Decode” and a middleline “Graphics Plane Access” in each chart indicate two operations thatare performed simultaneously when reproducing. The above algorithm isdescribed assuming that these two operations are performed in parallel.

The Graphics Plane Access includes a clearing period (1) and a writeperiod (3). The clearing period (1) indicates either a time periodnecessary to clear an entire Graphics Plane (90,000×(size of GraphicsPlane//256,000,000)), or a time period necessary to clear all Windows onthe Graphics Plane (Σ(90,000×(size of Window [i]//256,000,000)).

The write period (3) indicates a time period necessary to render anentire Window (90,000×(size of Window [i]//256,000,000)).

Further, a decode period (2) indicates a time period between the DTS andthe PTS of the ODS.

Lengths of the clearing period (1), the decode period (2), and the writeperiod (3) may vary depending on a range to be cleared, a size of ODS tobe decoded, and a size of the Graphics Object to be written to theGraphics Plane. For convenience, a starting point of the decode period(2) in the drawing is the same as a starting point of the clearingperiod (1).

FIG. 21B illustrates a case in which the decode period (2) is long, andthe decode_duration equals to a total of the decode period (2) and thewrite period (3).

FIG. 21C illustrates a case in which the clearing period (1) is long,and the decode_duration equals to a total of the clearing period (1) andthe write period (3).

FIGS. 22A to 22C illustrate a case in which two ODS is included in oneWindow. The decode period (2) in both FIGS. 22B and 22C indicates atotal time period necessary for decoding two Graphics. Likewise, thewrite period (3) indicates a total time period necessary for writing twoGraphics to the Graphics Plane.

Even though the number of ODS is two, it is possible to calculate thedecode_duration in the same manner as in the case of FIG. 21. When thedecode period (3) for decoding the two ODS is long, the decode_durationequals to a total of the decode period (2) and the write period (3) asshown in FIG. 22B.

When the clearing period (1) is long, the decode_duration equals to atotal of the clearing period (1) and the write period (3).

FIG. 23A describes a case in which each of two Windows includes an ODS.As in the previous cases, when the clearing period (1) is longer thanthe decode period (3) for decoding the two ODS, the decode_durationequals to a total of the clearing period (1) and the write period (3).However, when the clearing period (1) is shorter than the decode period(3), it is possible to write to a first Window before the decode period(2) ends. Accordingly, the decode_duration does not equal to either of atotal of the clearing period (1) and the write period (3), or a total ofthe decode period (2) and the write period (3).

When a time period necessary for decoding a first ODS is a write period(31) and a time period necessary for decoding a second ODS is a writeperiod (32), FIG. 23B illustrates a case in which the decode period (2)is longer than a total of the clearing period (1) and the write period(31). In this case, the decode_duration equals to a total of the decodeperiod (2) and the write period (32).

FIG. 23C illustrates a case in which a total of the clearing period (1)and the write period (31) is longer than the decode period (2). In thiscase, the decode_duration equals to a total of the clearing period (1),the write period (31), and the write period (32).

The size of the Graphics Plane is known from a model of the reproductionapparatus in advance. Also, the size of the Window, and the size andnumber of the ODS are also known at the authoring. Accordingly, it ispossible to find to which combination of time periods thedecode_duration equals: the clearing period (1) and the write period(3), the decode period (2) and the write period (3), the decode period(2) and the write period (32), or the clearing period (1), the writeperiod (3) and the write period (32).

By setting the PTS of the ODS based on the calculation of thedecode_duration explained above, it is possible to synchronously displaythe graphics with the picture data at a high accuracy. Such a syncdisplay at a high accuracy becomes possible by defining the Window andlimiting an area to re-draw to the Window. Thus, introducing a conceptof Window into an authoring environment has a great significance.

The following is an explanation about settings of the DTS and PTS of theWDS in the DSn. The DTS of the WDS may be set so as to satisfy anequation below. DTS(DSn[WDS])≧DTS(DSn[PCS])

On the other hand, the OTS of the WDS in the DSn indicates a-deadline tostart writing to the Graphics Plane. Because it is sufficient to writeto the Window on the Graphics Plane, the time to start writing to theGraphics Plane is determined by subtracting a time length indicated bythe PTS of the PCS from a time period necessary for writing the WDS.When a total size of the WDS is ΣSIZE(WDS.WIN[i]), the time necessaryfor clearing and re-drawing is “ΣSIZE (WDS.WIN[i])//256,000,000”. Whenexpressing at a time accuracy of 90.000 KHz, the time is“90,000×ΣSIZE(WDS.WIN[i])//256,000,000”.

Accordingly, it is possible to calculate the PTS of the WDS by thefollowing equation.PTS(DSn[WDS])=PTS(DSn[PCS])−90000×ΣSIZE(WDS.WIN[i])//256,000,000

The PTS indicated in the WDS is the deadline, and it is possible tostart writing to the Graphics Plane earlier than the PTS. In otherwords, as shown in FIG. 23, once decoding the ODS to be rendered in oneof the Windows, writing of the Graphics Object obtained by the decodingmay start at this point.

As described above, it is possible to assign the Window to any point oftime on the reproduction timeline of the AVClip using the DTS and PTSadded to the WDS.

Explanations about an example of settings of the DTS and PTS in aDisplay Set based on the settings are give below, referring to specificexample illustrated in FIGS. 24-25. The example is about a case in whichsubtitles are displayed by writing to the Graphics Plane four times, andan update is performed for displaying each of two subtitles “what isblu-ray.” and “blu-ray is everywhere.” FIG. 24 illustrates chronologicaltransition of the update in the example. Until a point t1, “what” isdisplayed, and “what is” is displayed after the t1 till a t2, and then“what is blu-ray.” is displayed at a t3. After a whole sentence of afirst subtitle has appeared, a second subtitle “blu-ray is everywhere.”is displayed at a t4.

FIG. 25A illustrates four Display Sets that are described so as toperform the above explained update. A DS1 includes a PCS1.2 forcontrolling an update at the t1, a PDS1 for coloring, an ODS1corresponding to the subtitle “what is blu-ray.”, and an END as anending code of the DS1.

A DS2 includes a PCS1.2 for controlling an update at the t2, and an END.A DS3 includes a PCS1.3 for controlling an update at a t3 and an END. ADS 4 includes a PCS2 for controlling an update at the t2, a PDS2 forcolor conversion, an ODS2 corresponding to the subtitle “blu-ray iseverywhere.”, and an END.

Referring to a timing chart in FIG. 25B, settings of DTS and PTS foreach functional segment in the four Display Sets are explained.

The reproduction timeline in the timing chart is the same as thetimeline in FIG. 24. In the timing chart of FIG. 25A, PTS(PCS1.1),PTS(PCS1.2), PTS(PCS1.3), and PTS(PCS2) are respectively set at adisplay point t1 for displaying “what”, a display point t2 fordisplaying “what is”, a display point t3 for displaying “what isblu-ray.”, and a display point t4 for displaying “blue-ray iseverywhere.”. Each PTS are set as above, because it is necessary thatthe control such as cropping described in each PCS is performed at thedisplay point of each subtitle.

PTS(ODS1) and PTS(ODS2) are set so as to indicate points that arecalculated by subtracting decode_duration from the points indicated byPTS (PCS1.1) and PTS (PCS2), respectively, because PTS(PCS) is requiredto be set so as to satisfy a formula below.PTS(DSn[PCS])≧DTS(DSn[PCS])+decode duration(DSn)

In FIG. 25B, PTS(ODS2) is set so as to indicate a point t5 that comesbefore the point t4, and PTS(ODS1) is set so as to indicate a point t0that comes before the point t1.

DTS (ODS1) and DTS (ODS2) are set so as to indicate points that arecalculated by subtracting decode_duration from the points indicated byPTS (ODS1) and PTS (ODS2), respectively, because DTS(ODS) is required tobe set so as to satisfy an equation below.PTS(DS[ODSj])=DTS(DSn[ODSj])+90,000×(SIZE(DSn[ODSj])//Rd

In FIG. 25B, PTS (ODS2) is set so as to indicate the point t5 that comesbefore the point t0, and PTS(ODS1) is set so as to indicate a point thatcomes before the point t0. A relation indicated by DTS (ODS2)=PTS (ODS1)is satisfied here.

By setting a PTS of an ODS immediately after a PTS of a preceding ODS tobe displayed earlier, the reproduction apparatus performs an operationin which the ODS is read out to the memory so as to overwrite thepreceding ODS, and thus it is possible that the reproduction process isperformed by a small size of memory. By realizing such a reproductionprocess, choices for a memory size for a reproduction apparatus becomewider.

The DTS of PCS1.1 is set so as to be DTS(PCS1.1)=DTS(ODS1), because thevalue for the DTS of PCS1.1 may be any point before the point indicatedby DTS(ODS1).

The PTS of ODS1, the DTS of ODS2, and the PTS of the PCS1.2, PCS1.3, andPCS2 are set at the point to, so as to satisfy a relation indicated byan equation below.PTS(ODS1)=DTS(ODS2)=PTS(PCS1.2)=PTS(PCS1.3)=PTS(PCS2)

This is because the value for the DTS of PCS1.2 and PCS1.3 may be anypoints before the point indicated by PTS (PCS1.3), and the DTS of PCS2may be any point before the point indicated by DTS(PCS2).

As explained above, it is possible to perform update of a succeeding PCSas soon as the updating of a previous PCS is completed, by reading outmore than one PCS at the same time.

It is sufficient that the DTS and PTS of PCS and the DTS and PTS of ODSsatisfy the relations indicated by the formulae above. Accordingly, itbecomes possible that the values are set to be DTS(ODS2)=PTS(ODS1) orPTS(ODS1)=DTS(ODS2)=PTS(PCS1.2)=PTS(PCS1.3)=DTS(PCS2). By such settingsfor time stamps, it is possible to adjust time length of a period inwhich load in decoding increases or more buffers are needed. Suchadjustment expands possibility of the controls during the reproduction,and advantageous for those who perform authoring or manufacturereproducing apparatuses.

Data structures of the Display Sets (PCS, WDS, PDS, ODS) explained aboveis an instance of the class structure described in a programminglanguage. Producers that perform authoring may obtain the datastructures on the BD-ROM by describing the class structure according tothe syntax provided in the Blu-ray Disc Prerecording Format.

Next, a practical example of a reproduction apparatus according to thepresent invention is explained below. FIG. 26 illustrates an internalstructure of the reproduction apparatus according to the presentinvention. The reproduction apparatus according to the present inventionis industrially produced based on the internal structure shown in thedrawing. The reproduction apparatus according to the present inventionis mainly structured by three parts: a system LSI, a drive device, and amicrocomputer system, and it is possible to industrially produce thereproduction apparatus by mounting the three parts to a cabinet and asubstrate of the apparatus. The system LSI is an integrated circuit inwhich various processing units for carrying out a function of thereproduction apparatus are integrated. The reproduction apparatusmanufactured in the above manner comprises a BD drive 1, a Read Buffer2, a PID filter 3, Transport Buffers 4 a-4 c, a peripheral circuit 4 d,a Video Decoder 5, a Video Plane 6, an Audio Decoder 7, a Graphics Plane8, a CLUT unit 9, an adder 10, a Graphics Decoder 12, a Coded DataBuffer 13, a peripheral circuit 13 a, a Stream Graphics Processor 14, anObject Buffer 15, a Composition Buffer 16, and a Graphics controller 17.

The BD drive 1 performs load/read/eject of the BD-ROM, and accesses tothe BD-ROM.

The Read Buffer 2 is a FIFO memory for storing the TS packets read fromthe BD-ROM in a first-in first-out order.

The PID filter 3 filters more than one TS packet outputted from the ReadBuffer 2. The filtering by the PID filter 3 is to write the only TSpackets having a desired PID to the Transport Buffers 4 a-4 c. Bufferingis not necessary for the filtering by the PID filter 3, and accordingly,the TS packets inputted to the PID filter 3 are written to the TransportBuffers 4 a-4 c without delay.

The Transport Buffers 4 a-4 c are for storing the TS packets outputtedfrom the PID filter 3 in a first-in first-out order. A speed at whichthe TS packets from the Transport Buffers 4 a-4 c are outputted is aspeed Rx.

The peripheral circuit 4 d is a wired logic for converting the TSpackets read from the Transport Buffers 4 a-4 c into functionalsegments. The functional segments obtained by the conversion are storedin the Coded Data Buffer 13.

The Video Decoder 5 decodes the more than one TS packets outputted fromthe PID filter 3 into a decompressed picture and writes to the VideoPlane 6.

The Video Plane 6 is a plane memory for a moving picture.

The Audio Decoder 7 decodes the TS packets outputted from the PID filter3 and outputs decompressed audio data.

The Graphics Plane 8 is a plane memory having an area for one screen,and is able to store decompressed graphics for one screen.

The CLUT unit 9 converts an index color of the decompressed Graphicsstored in the Graphics Plane 8 based on the values for Y, Cr, and Cbindicated by the PDS.

The adder 10 multiplies the decompressed Graphics to which the colorconversion has been performed by the CLUT unit 9 by the T value(Transparency) indicated by the PDS, adds the decomposed picture datastored in the Video Plane per pixel, then obtains and outputs thecomposed image.

The Graphics Decoder 12 decodes the Graphics Stream to obtain thedecomposed graphics, and writes the decomposed graphics as the GraphicsObject to the Graphics Plane 8. By decoding the Graphics Stream, thesubtitles and menus appear on the screen. The Graphics Decoder 12includes the Coded Data Buffer 13, the peripheral circuit 13 a, theStream Graphics Processor 14, the Object Buffer 15, the CompositionBuffer 16, and the Graphics controller 17.

The Coded Data Buffer 13 is a buffer in which the functional segment isstored along with the DTS and PTS. The functional segment is obtained byremoving a TS packet header and a PES packet header from each TS packetin the Transport Stream stored in the Transport Buffer 4 a-4 c and byarranging the payloads sequentially. The PTS and DTS out of the removedTS packet header and PES packet header are stored after makingcorrespondence between the PES packets.

The peripheral circuit 13 a is a wired logic that realizes a transferbetween the Coded Data Buffer 13 and the Stream Graphics Processor 14,and a transfer between the Coded Data Buffer 13 and the CompositionBuffer 16. In the transfer operation, when a current time is a timeindicated by the DTS of the ODS, the ODS is transferred from the CodedData Buffer 13 to the Stream Graphics Processor 14. When the currenttime is a time indicated by the DTS of the PCS and PDS, the PCS and PDSare transferred to the Composition Buffer 16.

The Stream Graphics Processor 14 decodes the ODS, and writes thedecompressed graphics of the index color obtained by decoding as theGraphics Object to the Object Buffer 15. The decoding by the StreamGraphics Processor 14 starts at the time of the DTS corresponding to theODS, and ends by the decode end time indicated by the PTS correspondingto the ODS. The decoding rate Rd of the Graphics Object is an outputrate of the Stream Graphics Processor 14.

The Object Buffer 15 is a buffer corresponding to a pixel buffer in theETSI EN 300 743 standard, and the Graphics Object obtained by the decodethat the Stream Graphics Processor. 14 performs is disposed. The ObjectBuffer 15 needs to be set to twice or four times as large as theGraphics Plane 8, because in case the Scrolling effect is performed, theObject Buffer 15 needs to store the Graphics Object that is twice orfour times as large as the Graphics Plane.

The Composition Buffer 16 is a memory in which the PCS and PDS aredisposed.

The Graphics controller 17 decodes the PCS disposed in the CompositionBuffer 16, and performs a control based on the PCS. A timing forperforming the control is based on the PTS attached to the PCS.

Next, recommended values for the transfer rate and buffer size forstructuring the PID filter 3, Transport Buffer 4 a-4 c, Graphics Plane8, CULT unit 9, Coded Data Buffer 13′, and Graphic's controller 17 areexplained. FIG. 27 illustrates sizes of the write rates Rx, Rc, and Rd,Graphics Plane 8, Coded Data Buffer 13, Object Buffer 15, andComposition Buffer 16.

The transfer rate Rc between the Object Buffer 15 and the Graphics Plane8 is the highest transfer rate in the reproduction apparatus of thepresent embodiment, and calculated as 256 Mbps (=500 Kbytes×29.97×2)from the window size and the frame rate.

Unlike the Rc, the transfer rate Rd (Pixel Decoding Rate) between theStream Graphics Processor 14 and Object Buffer 15 does not need to beupdated every video frame cycle, and ½ or ¼ of the Rc is sufficient forthe Rd. Accordingly, the Rd is either 128 Mbps or 64 Mbps.

The Transport Buffer Leak Rate Rx between the Transport Buffer 4 a-4 cand Coded Data Buffer 13 is a transfer rate of the ODS in a compressedstate. Accordingly, the transfer rate Rd multiplied by the compressionrate is sufficient for the Transport Buffer leak rate Rx. Given thecompression rate of the ODS is 25%, 16 Mbps (=64 Mbps×25%) issufficient.

The transfer rates and buffer sizes shown in the drawing are the minimumstandard, and it is also possible to set at higher rates and largersizes.

In the above structured reproduction apparatus, each elements perform adecoding operation in a pipeline structure.

FIG. 28 is a timing chart illustrating a pipeline processing by thereproduction apparatus. A 5th row in the drawing is a Display Set in theBD-ROM, a 4th row shows read periods from the PCS, WDS, PDS, and ODS tothe Coded Data Buffer 13. A 3rd row shows decode periods of each ODS bythe Stream Graphics Processor 114. A 1st row shows operations that theGraphics controller 17 performs.

The DTS (decode starting time) attached to the ODS1 and ODS2 indicatet31 and t32 in the drawing, respectively. Because the decode startingtime is set by DTS, each ODS is required to be read out to the CodedData Buffer 13. Accordingly, the reading of the ODS1 is completed beforea decode period dp1 in which the ODS1 is decoded to the Coded DataBuffer 13. Also, the reading of the ODS2 is completed before a decodeperiod dp2 in which the ODS2 is decoded to the Coded Data Buffer 13.

On the other hand, the PTS (decode ending time) attached to the ODS1 andODS2 indicate t32 and t33 in the drawing, respectively. Decoding of theODS1 by the Stream Graphics Processor 14 is completed by the t32, anddecoding of the ODS2 is completed by a time indicated by the t33. Asexplained above, the Stream Graphics Processor 14 reads the ODS to theCoded Data Buffer 13 by the time the DTS of the ODS indicates, anddecodes the ODS read to the Coded Data Buffer 13 by the time the PTS ofthe ODS indicates, and write the decoded ODS to the Object Buffer 15.

A period cd1 at the 1st row in the drawing indicates a period necessaryfor the Graphics controller 17 to clear the Graphics Plane. Also, aperiod td1 indicates a period necessary to write the Graphics Objectobtained on the Object Buffer to the Graphics Plane 8. The PTS of theWDS indicates the deadline to start writing, and the PTS of the PCSindicates ending of the write and a timing for display. At the timeindicated by the PTS of the PCS, the decompressed graphics to compose aninteractive screen is obtained on the Graphics Plane 8.

After the CLUT unit 9 performs the color conversion of the decompressedgraphics and the adder 10 performs composition of the decomposedgraphics and a decomposed picture stored in the Video Plane 6, acomposite image is obtained.

In the Graphics Decoder 12, the Stream Graphics Processor 14 performsdecoding continuously while the Graphics controller 17 performs clearingof the Graphics Plane 8. By the above pipeline processing, it ispossible to perform a prompt display of the graphics.

In FIG. 28, a case in which the clearing of the Graphics Plane endsbefore completing the decoding of the ODS is explained. FIG. 29illustrates a timing chart in a pipeline processing of a case in whichthe decoding of the ODS ends before the clearing of the Graphics Planeis completed. In this case, it is not possible to write to the GraphicsPlane at a time of completion of the decoding of the ODS. When theclearing of the Graphics Plane is completed, it becomes possible towrite the graphics obtained by the decode to the Graphics Plane.

Next, a chronological transition in buffer occupancy is explained. FIG.30 is a timing chart showing the chronological transitions of thefollowing components shown in FIG. 26: the composition buffer 16, theobject buffer 16, the coded data buffer 13, and the Graphics Plane 8.The first to fourth rows illustrate the chronological transitions inoccupancy of the Graphics Plane 8, the object buffer 15, the coded databuffer 13, and the composition buffer 16, respectively. Here, achronological transition is described using line graph, in which thelateral axis represents timeline, and the longitudinal axis representsoccupancy.

The fourth row of FIG. 30 shows the chronological transition inoccupancy for the composition buffer 16. As the forth row shows, thechronological transition for the composition buffer 16 includes a part“vfo”, which represents simple increase which is due to storage of PCSoutputted from the coded data buffer 13.

The third row shows the chronological transition in occupancy for thecoded data buffer 13. As the third row shows, the chronologicaltransition for the coded data buffer 13 includes two simply increasingparts vf1 and vf2, and two simply decreasing parts vg1 and vg2. Thegradients of the simply increasing parts vf1 and vf2 depend on theoutput rate Rx of the transport buffers 4 a,b,c to the coded data buffer13, and the gradients of the simply decreasing parts vg1 and vg2represent decoding performed by the stream graphics processor 14, whichis performed in an instant. In other words, decoding for ODS isperformed instantly, and the stream graphics processor 14 retains thedecompressed graphics obtained by the decoding. The write rate of thetransmission path from the stream graphics processor 14 to the objectbuffer 15 is 128 Mbps. Therefore the occupancy of the object buffer 15increases in accordance with this write rate.

The second row represents the chronological transition in occupancy forthe object buffer 15. As the second row shows, the chronologicaltransition for the object buffer 15 includes simply increasing parts vh1and vh2, which is due to storage of ODS outputted from the streamgraphics processor 14. The gradients of the simply increasing parts Vh1and Vh2 depend on the transfer rate Rc from the stream graphicsprocessor 14 to the object buffer 15. The periods during which simplydecreasing parts at the third row and the simply increasing parts at thesecond row occur correspond to “decode period”. The start of such adecode period is indicated by the DTS of the ODS, and the end of thedecode period is indicated by the PTS of the ODS. If the decompressedgraphics is stored in the object buffer 15 until the time shown by theDTS of the ODS, that means the completion of decoding directed to theODS. As long as the decompressed graphics is stored in the object buffer15 until the time shown by the PTS of the ODS, the simply increasingparts and the simply decreasing parts, during this decode period, maytake any form.

The first row represents the chronological transition in occupancy forthe Graphics Plane 8. As the first row shows, the chronologicaltransition for the Graphics Plane 8 includes a simply increasing partvf3, which is due to storage of already decoded ODS outputted from theObject Buffer 115. The gradient of the simply increasing part Vf3depends on the transfer rate Rd from the Object Buffer 15 to theGraphics Plane 8. The end of the simply increasing part is shown by thePTS of ODS.

The graphs such as in FIG. 27 are illustrated using: DTS and PTSassigned to ODS; DTS and PTS assigned to ICS; size and transfer rate ofeach buffer illustrated in FIG. 27. Furthermore, by creating the graphssuch as in this diagram, users can know how the state of each bufferchanges, at the authoring stage.

Since the transition of the state of each buffer can be adjusted byupdating DTS and PTS, it becomes possible to avoid decoding load, whichwould exceed the specification of the decoder, from being imposed on thereproduction apparatus, and to avoid buffer overflow incident toreproduction. According to this, the hardware/software implementationwill become easy, in the developing phase of reproduction apparatuses.

Next, how the controlling unit 20 and the Graphics Decoder 12 areimplemented is explained below. The controlling unit 20 is implementedby writing a program performing an operation shown in FIG. 30, andhaving a general CPU to execute the program. The operation performed bythe controlling unit 20 is explained by referring to FIG. 30.

FIG. 31 is a flowchart showing the process of loading operation offunctional segment. In the flowchart, SegmentK is a variable indicatingeach of Segments (PCS, WDS, PDS, and ODS) that is read out inreproducing the AVClip. An ignore flag is a flag to determine if theSegmentK is ignored or loaded. The flowchart has a loop structure, inwhich first the ignore flag is initialized to 0 and then Steps S21-S24and Steps S27-S31 are repeated for each SegmentK (Step S25 and StepS26).

Step S21 is to judge if the SegmentK is the PCS, and if the SegmentK isthe PCS, judgments in Step S27 and Step S28 are performed.

Step S22 is to judge if the ignore flag is 0. If the ignore flag is 0,the operation moves to Step S23, and if the ignore flag is 1, theoperation moves to Step S24. If the ignore flag is 0 (Yes in Step S22),the SegmentK is loaded to the Coded Data Buffer 13 in Step S23.

If the ignore flag is 1 (No in Step S22), the SegmentK is ignored inStep S24. By this, the rest of all functional segments that belong tothe DS are ignored because Step S22 is No (Step S24).

As explained above, whether the SegmentK is ignored or loaded isdetermined by the ignore flag. Steps S27-S31, S34, and S35 are steps forsetting the ignore flag.

In Step S27, it is judged if segmet_type of the SegmentK is theAcquisition Point. If the SegmentK is the Acquisition Point, theoperation moves to Step S28, and if the SegmentK is either the EpochStart or Normal Case, then the operation moves to Step S31.

In Step S28, it is judged if a preceding DS exists in any of the buffersin the Graphics Decoder 12 (the coded data buffer 13, stream graphicsprocessor 14, object buffer 15, and composition buffer 16). The judgmentin Step S28 is made when the judgment in Step S27 is Yes. A case inwhich a preceding DS does not exist in the Graphics Decoder 12 indicatesa case in which the skip operation is performed. In this case, thedisplay starts from the DS that is the Acquisition Point, and thereforethe operation moves to Step S30 (No in Step S28). In Step S30, theignore flag is set to 0 and the operation moves to Step S22.

A case in which a preceding DS exists in the Graphics Decoder 12indicates a case in which normal reproduction is performed. In thiscase, the operation moves to Step S29 (Yes in Step S28). In Step S29,the ignore flag is set to 1 and the operation moves to Step S22.

In Step S31, it is judged if composition_state of the PCS is the NormalCase. If the PCS is the Normal Case, the operation moves to Step S34,and if the PCS is the Epoch Start, then the ignore flag is set to 0 inStep S30.

In Step S34, like in Step S28, it is judged if a preceding DS exists inany of the buffers in the Graphics Decoder 12. If the preceding DSexists, the ignore flag is set to 0 (Step S30). If the preceding DS doesnot exist, it is not possible to obtain sufficient functional segmentsto compose an interactive screen and the ignore flag is set to 1 (StepS35).

By setting the ignore flag in the above manner, the functional segmentsthat constitute the Normal Case are ignored when the preceding DS doesnot exit in the Graphics Decoder 12.

Taking an example of a case in which the DS is multiplexed as shown inFIG. 31, a manner how the reading of the DS is performed is explained.In the example of FIG. 31, three DS are multiplexed with a movingpicture. The composition_state of a DS1 is Epoch Start, thecomposition_state of a DS10 is Acquisition Point, and thecomposition_state of a DS20 is Normal Case.

Given that, in an AVClip in which the three DS and the moving pictureare multiplexed, a skip operation to a picture data pt10 as shown by anarrow am1 is performed, the DS10 is the closest to a skipping target,and therefore the DS10 is the DS described in the flowchart in FIG. 30.Although the composition_state is judged to be the Acquisition Point inStep S27, the ignore flag is set to 0 because no preceding DS exists inthe Coded Data Buffer 13, and the DS10 is loaded to the Coded DataBuffer 13 of the reproduction apparatus as shown by an arrow md1 in FIG.32. On the other hand, in a case in which the skipping target is afterthe DS10 (an arrow am2 in FIG. 31), the DS20 is to be ignored becausethe DS20 is Normal Case Display Set and DS20 because a preceding DS doesnot exist in the Coded Data Buffer 13 (an arrow md2 in FIG. 32).

FIG. 33 illustrates loading of the DS1, DS10, and DS20 in a normalreproduction. The DS1 whose composition_state of the PCS is the EpochStart is loaded to the Coded Data Buffer 13 as it is (Step S23).However, because the ignore flag of the DS10 whose composition_state ofthe PCS is the Acquisition Point is set to 1 (Step S29), the functionalsegments that constitute the DS10 are ignored and not loaded to theCoded Data Buffer 13 (an arrow rd2 in FIG. 34, and Step S24). Further,the DS20 is loaded to the Coded Data Buffer 13, because thecomposition_state of the PCS of the DS20 is the Normal Case (an arrowrd3 in FIG. 34).

Next, operations by the Graphics controller 17 are explained. FIGS.35-37 illustrate a flowchart showing the operations performed by theGraphics controller 17.

Steps S41-S44 are steps for a main routine of the flowchart and waitsfor any of events prescribed in Steps S41-S44 occurs.

Step S41 is to judge if a current reproducing time is a time indicatedby the DTS of the PCS, and if the judging is Yes, then an operation inSteps S45-S53 is performed.

Step S45 is to judge if the composition_state of the OCS is theepoch_start, and if judged to be the epoch_start, the Graphics Plane 8is all cleared in Step S46. If judged to be other than the epoch_start,the Window indicated by the window_horizontal_position,window_vertical_position, window_width, and window_height of the WDS iscleared.

Step S48 is a stepper formed after the clearing performed in Step S46 orin Step S47, and to judge if the time indicated by the PTS of any ODSxhas passed. The decoding of any ODSx could be already completed by thetime the clearing ends, because the clearing of an entire Graphics Plane8 takes time. Therefore, in Steps S48, it is judged if the decoding ofany ODSx is already completed by the time the clearing ends. If thejudging is No, the operation returns to the main routine. If the timeindicated by the PTS of any ODSx has already passed, an operation inSteps S49-S51 is performed. In Step S49, it is judged ifobject_crop_flag is 0, and if the flag indicates 0, then the GraphicsObject is set to “no display” (Step S50).

If the flag is not 0 in Step S49, then an object cropped based onobject_cropping_horizontal_position, object_cropping_vertical_position,cropping_width, and cropping_height is written to the Window in theGraphics Plane 8 at the position indicated byobject_cropping_horizontal_position andobject_cropping_vertical_position (Step S51). By the above operation,one or more Graphics Objects are rendered in the Window.

In Step 52, it is judged if the time corresponding to a PTS of anotherODSy has passed. When writing the ODSx to the Graphics Plane 8, if thedecoding of the ODSy has already been completed, then the ODSy becomesODSx (Step S53), and the operation moves to Step S49. By this, theoperation from Steps S49-S51 is also performed to another ODS.

Next, by referring to FIG. 36, Step S42 and Steps S54-S59 are explainedbelow.

In Step 42, it is judged if the current reproducing point is at the PTSof the WDS. If the judging is that the current reproducing point is atthe PTS of the WDS, then it is judged if the number of the Window is oneor not in Step S54. If the judging is two, the operation returns to themain routine. If the judging is one, a loop processing of Steps S55-S59is performed. In the loop processing, operations in Steps S55-S59 areperformed to each of the two Graphics Object displayed in the Window. InStep S57, it is judged if object_crop_flag indicates 0. If it indicates0, then the Graphics is not displayed (Step S58).

If it does not indicate 0, then a cropped object based onobject_cropping_horizontal_position, object_cropping vertical_position,cropping_width, and cropping_height is written to the Window in theGraphics Plane 8 at the position indicated byobject_cropping_horizontal_position andobject_cropping_vertical_position (Step S59). By repeating the aboveoperations, more than one Graphics Object is rendered in the Window.

In Step S44, it is judged if the current reproducing point is at the PTSof the PDS. If the judging is that the current reproducing point is atthe PTS of the PDS, then it is judged if pallet_update_flag is one ornot in Step S60. If the judging is one, the PDS indicated by pallet_idis set in the CLUT unit (Step S61). If the judging is 0, then Step S61is skipped.

After that, the CLUT unit performs the color conversion of the GraphicsObject on the Graphics Plane 8 to be combined with the moving picture(Step S62).

Next, by referring to FIG. 37, Step S43 and Steps S64-S66 are explainedbelow.

In Step 43, it is judged if the current reproducing point is at the PTSof the ODS. If the judging is that the current reproducing point is atthe PTS of the ODS, then it is judged if the number of the Window is twoor not in Step S63. If the judging is one, the operation returns to themain routine. If the judging is two, operations in Steps S64-S66 areperformed. In Step S64, it is judged if object_crop_flag indicates 0. Ifit indicates 0, then the Graphics is not displayed (Step S65).

If it doesn't indicate 0 then a cropped object based onobject_cropping_horizontal_position, object_cropping vertical_position,cropping_width, and cropping_height is written to the Window in theGraphics Plane 8 at the position indicated byobject_cropping_horizontal_position andobject_cropping_vertical_position (Step S66). By repeating the aboveoperations, the Graphics Object is rendered in each Window.

The above explanations are about the DTS and PTS of the PCS, and the DTSand PTS of the ODS that belong to DSn. The DTS and PTS of the PDS, andthe DTS and PTS of the END are not explained. First, the DTS and PTS ofthe PD that belongs to the DSn are explained.

As for the PDS that belongs to the DSn, it is sufficient if the PDS isavailable in the CLUT unit 9 by the PCS is loaded to the CompositionBuffer 16 (DTS(DSn[PCS])) after decoding start point of a first ODS(DTS(DSn[ODS1])). Accordingly, a value of PTS of each PDS (PDS1-PDSlast)in the DSn is required to be set so as to satisfy the followingrelations.DTS(DSn[PCS])≦PTS(DSn[PDS1])PTS(DSn[PDSj])≦PTS(DSn[PDSj+1])≦PTS(DSn[PDSlast])PTS(DSn[PDSlast])≦DTS(DSn[ODS1])

Note that the DTS of the PDS is not referred to during the reproducing,the DTS of the ODS is set to the same value as the PTS of the PDS inorder to satisfy the MPEG2 standard.

Following is an explanation about roles of the DTS and PTS in thepipeline processing of the reproduction apparatus when the DTS and PDSare set so as to satisfy the above relations. FIG. 38 illustrates thepipeline of the reproduction apparatus based on the PTS of the PDS. FIG.38 is based on FIG. 26. A first row in FIG. 38 indicates setting the ODSin the CLUT unit 9. Under the first row are the same as first to fifthrows in FIG. 26. The setting of PDS1-last in the CLUT unit 9 isperformed simultaneously with the start of the ODS1 decoding (arrowsup2, up3). (The setting of the PDS1-PDSlast to the CLUT unit 9 isperformed after the transferring the PCS and WDS and before the decodingof the ODS1, and accordingly the setting of the PDS1-PDSlast to the CLUTunit 9 is set before a point indicated by the DTS of the ODS1 as shownby arrows up2 and up3.)

As described above, the setting of the PDS is performed in prior to thedecoding of the ODS.

Next, a setting of the PTS of END of Display Set segment in the DSn isexplained. The END that belongs to the DSn indicates the end of the DSn,and accordingly it is necessary that the PTS of the END indicates thedecode ending time of the ODS2. The decode ending time is indicated bythe PTS(PTS(DSn[ODSlast])) of the ODS2(ODSlast), and therefore the PTSof the END is required to be set at a value that satisfies an equationbelow.PTS(DSn[END])=PTS(DSn[ODSlast])

In terms of a relation between the DSn and the PCS that belongs to theDSn+1, the PCS in the DSn is loaded to the Composition Buffer 16 beforea loading time of the first ODS (ODS1), and therefore the PTS of the ENDshould be after a loading time of the PCS in the DSn and before aloading time of the PCS that belongs to the DSn+1. Accordingly, the PTSof the END is required to satisfy a relation below.DTS(DSn[PCS])≦PTS(DSn[END])≦DTS(DSn+1[PCS])

On the other hand, the loading time of the first ODS (ODS1) is before aloading time of a last PDS (PDSlast), and therefore the PTS of the END(PTS (DSn [END])) should be after a loading time of the PDS that belongsto the DSn (PTS(DSn[PDSlast])). Accordingly, the PTS of the END isrequired to satisfy a relation below.PTS(DSn[PDSlast])≦PTS(DSn[END])

Following is an explanation about significance of the PTS of the END inthe pipeline processing of the reproduction apparatus. FIG. 39 is adiagram describes the significance of the END in the pipeline process ofthe reproduction apparatus FIG. 39 is based on FIG. 28, and each row inFIG. 39 is substantially the same as FIG. 28 other than that a first rowin FIG. 39 indicates the content of the Composition Buffer 16. Further,in FIG. 39, 2 Display Sets, DSn and DSn+1 are illustrated. The ODS lastin the DSn is the last ODSn of A-ODSs, and accordingly, the pointindicated by the PTS of the END is before the DTS of the PCS in theDSn+1.

By the PTS of the END, it is possible to find when the loading of theODS in the DSn is completed during reproduction.

Note that although the DTS of the END is not referred to duringreproduction, the DTS of the END is set to the same value as the PTS ofthe END in order to satisfy the MPEG2 standard.

As described in the above, a part of the Graphics Plane is specified asthe Window for displaying the Graphics according to the presentembodiment, and therefore the reproduction apparatus does not have torender the Graphics for an entire Plane. The reproduction apparatus mayrender the Graphics for only a predetermined size of Window, 25% to 33%of the Graphics Plane, for example. Because the rendering of theGraphics other than the Graphics in the Window is not necessary, theload for software in the reproduction apparatus decreases.

Even in a worst case in which the updating of the Graphics is performed¼ of the Graphics Plane, for example, it is possible to display theGraphics synchronously with the picture by the reproduction apparatusperforming the write to the Graphics Plane at a predetermined transferrate such as 256 Mbps, and by setting the size of the Window so as toensure the sync display with the picture.

Thus, it is possible to realize a high resolution subtitle display forvarious reproduction apparatuses, because the sync display is easilyensured.

Second Embodiment

The First Embodiment explained above is for graphics dedicated forsubtitle display. On the contrary, a Second Embodiment is on graphicsfor interactive display.

Among the embodiments of a recording medium according to the presentinvention, an example of use of the recording medium is explained asfollows. Just as in the First Embodiment, the recording medium of theSecond Embodiment can also be manufactured by an improvement in anapplication layer of a BD-ROM. FIG. 41 is a diagram schematicallyillustrating a structure of the AVClip of the second embodiment.

The AVClip (shown in the middle) is structured in a following manner.The video stream made of plural video frames (picture pj1, pj2, andpj3), and the audio stream made of plural audio frames (top row of thedrawing) are respectively converted into a line of PES packets (secondrow of the drawing), and then into a line of TS packets (third row ofthe drawing). The interactive graphics stream (bottom row of thedrawing) is converted into a line of PES packets (second to bottom rowof the drawing), and then into a line of TS packets (third to bottom rowof the drawing). The three lines of TS packets are multiplexed, and thusthe AVClip is constituted.

Next, the interactive graphics stream is explained. The interactivegraphics stream has an Interactive Composition Segment (ICS) instead ofPCS, and does not have WDS. The interactive graphics stream is similarto the presentation graphics stream, in that it has functional segmentsnamed a Palette Definition Segment (PDS), an Object Definition Segment(ODS), and an END of Display Set Segment (END).

Disposition of GUI parts on a screen generates an interactive screenthat is defined by the functional segments. FIG. 42A is a diagramillustrating such an interactive screen realized by the interactivegraphics stream. This interactive screen includes four GUI parts calledbuttons A-buttons D. Interactivity by means of interactive graphicsstream means changing the state of these GUI parts (i.e. buttons)according to user operations. The state of the GUI parts (buttons)includes “normal state bt1”, “selected state bt2”, and “active statebt3”, which are shown in FIG. 42A. The normal state is a state in whichmere display is provided. As opposed to this, the selected state is astate in which a focus is given according to a user operation, butconfirmation has not been received. The active state is a state in whichconfirmation is received. The button state can be changed by pushdirected to the keys of the remote controller 400, which is shown in theFirst Embodiment.

FIG. 42B is a diagram illustrating the keys of the remote controller400, through which a user operation directed to the interactive screenis received. As shown in this drawing, the remote controller 400 isprovided with MoveUp key, MoveDown key, MoveRight key, and MoveLeft key.

The MoveUp key is for, when a botton in the interactive screen is in aselected state, setting a button above this selected button, to be in aselected state. The MoveDown key is for setting a button below thisselected button to be in a selected state. The MoveRight key is forsetting a key on the right of the selected key to be in a selectedstate, and the MoveLeft key is for setting a key on the left of theselected key to be in a selected state.

The Activated key is for setting the selected button to be in an activestate (i.e. to activate. Numerical keys “0”-“9” are for setting a buttonto which a corresponding number is assigned, to be in a selected state.“+10” key is for receiving an operation to add 10 to the alreadyinputted numerical values. It should be noted here that “0” key and“+10” key are both for receiving input of numerical value of no smallerthan 10 digits. Therefore any one of them is enough for the remotecontroller 400.

Each state (i.e. normal state, selected state, and active state) is madeof plural graphics in decompressed state. Each decompressed graphics,which is for representing each state of the buttons, is named “GraphicsObject”. The reason why one state of a button is represented by pluraldecompressed graphics is to take it into consideration to performanimation display for every state of each button.

Next, the improvements directed to the definition segments (ODS, PDS) inthe present embodiment are explained. ODS and PDS have the same datastructure as those of the First Embodiment. Only difference lies in the“object_ID” concerning ODS. ODS in the Second Embodiment constructsanimation using plural Graphics Objects defined by plural ODSs. Inconstructing animation, an object_ID is added to a series of ODS, wherethe object_ID is a serial number.

Next, ICS is explained. An Interactive Composition Segment is afunctional segment constituting an interactive screen. The InteractiveComposition Segment has the data structure shown in FIG. 43. As shown inthis drawing, ICS is composed of: segment_type; segment_length;composition_number; composition_state; command_update flag;composition_time_out_pts; selection_time_out_pts; UO_mask_table;animation_frame_rate_code; default_selected_button_number;default_activated_button_number; and button information sets (buttoninfo (1) (2) (3) . . . ).

“composition_number” represents numerical values from 0 to 15, whichindicate performing of update.

“composition_state” represents the DS starting with the present ICS isNormal Case, Acquisition Point, or Epoch Start.

“command_update_flag” represents whether the button command within thepresent ICS has changed from the former ICS. For example, if the DS thata certain ICS belongs is Acquisition Point, this ICS will have the samecontent as an ICS which is just before this ICS, in principle. However,if “command_update_flag” is set to be on, a button command differentfrom the ICS just before can be set as ICS. This flag is set to be validwhen command is desired to be changed while applying Graphics Object.

“compsotion_time_out_pts” describes an end time of the interactivescreen. At the end time, the interactive screen display is no longervalid, and so is not performed. It is desirable thatcomposition_time_out_pts is described at a time accuracy of thereproduction timeline for the moving picture data.

“selection_time_out_pts” describes an end time of a valid buttonselection period. During the selection_time_out_pts, the buttonspecified by default_activated_button_number gets activated. The periodof selection_time_out_pts is equal to, or shorter than the period ofcomposition_time_out_pts. Selection_time out_pts is described at a timeaccuracy of the video frame.

“UO_mask_table” represents permission/prohibition of user operation forDisplay Set that corresponds to the ICS. If this mask field is set asprohibition, use operation directed to the reproduction apparatus willbe invalid.

“animation_frame_rate_code” describes a frame rate to be applied to theanimation-type button. An animation frame rate is obtained by dividingthe video frame rate by a value in this field. If the value in thisfield is “00”, only the ODS specified by start_object_id_xxx isdisplayed, and not animation, the ODS being among the ODSs definingGraphics Objects for the buttons.

“default_selected_button_number” indicates a button number that shouldbe set in a selected state as a default, when an interactive screendisplay has started. If this field is “0”, a button having the buttonnumber stored in the register of the reproduction apparatus will beautomatically set to be in an active state. If this field is not “0”, itmeans that this field indicates a valid value of button.

“default_activated_button_number” represents a button to beautomatically set in an active state, when the user did not set anybutton to be in an active state prior to the time defined byselection_time_out_pts. If default_activated_button_number is “FF”, thebutton currently in a selected state, at the time defined byselection_time_out_pts, will be automatically selected. If thisdefault_activated_button_number is “00”, the automatic selection willnot be performed. If it is other than “00” and “FF”, this field will beinterpreted as indicating a valid button number.

Button information (button_info) defines each button composed at theinteractive screen. A leader line in this drawing focuses the internalstructure of button information i, which is about the ith button thatthe ICS controls. The following explains the information elementsconstituting the button information i.

“button_number” is a value uniquely identifying the button i, at theICS.

“numerically_selectable_flag” indicates whether to permit the numericalvalue selection for the button i.

“auto_action_flag” indicates whether to automatically set the button i.If this auto_action_flag is set to be on (i.e. bit value of 1), thebutton i will be set in an active state, instead of in a selected state.Conversely, if the auto_action_flag is set to be off (i.e. bit value of0), the button i will be set in a mere selected state, even if thisbutton has been selected.

“object_horizontal_position” and “object_vertical_position” respectivelyindicate the horizontal position and the vertical position, of theupper-left pixel of the button i in the interactive screen.

“upper_button_number” indicates the number of a button to be set in aselected state instead of the button i, when the MoveUp key has beenpushed at the time when the button i is in a selected state. If thenumber corresponding to the button i has been set in this field, thepush directed to the MoveUp key will be ignored.

“lower_button_number”, “left_button_number”, and “right_button_number”indicate, when MoveDown key, MoveLeft key, and MoveRight key arerespectively pushed during the button i being in a selected state, thenumber of a button to be in a selected state instead of the push of thebutton i. If the number corresponding to the button i has been set inthis field, the push directed to these keys will be ignored.

“start_object_id_normal” is such a field that, when rendering the buttoni in a normal state by animation, the first number among the serialnumbers that are assigned to the plurality of ODSs constituting theanimation is described in this “start_object_id_normal”.

“end_object_id_normal” is such a field that, when rendering the button iin a normal state by animation, the last number among the serial numbers(i.e. object_id) that are assigned to the plurality of ODSs constitutingthe animation is described in this end_object_id_normal. If the IDindicated in the end_object_id_normal is the same as the ID indicated inthe start_object_id_normal, the still image corresponding to this ID inthe Graphics Object will be the image for the button i.

“repeated_normal_flag” indicates whether to repeatedly continue theanimation display of the button i that is in a normal state.

“start_object_id_selected” is such a field that, when rendering thebutton i in a selected state in animation, the first number among theserial numbers that are assigned to the plurality of ODSs consitutitingthe animation is described in this “start_object_id_selected”.

“end_object_id_selected” is such a field that, when rendering the buttonin a selected state by animation, the last number among the serialnumbers that are assigned to the plurality of ODSs constituting theanimation is described in this end_object_id_selected. If the IDindicated in the the end_object_id_selected is the same as the IDindicated in the start_object_id_selected, the still image correspondingto this ID in the Graphics Object will be the image for the button i.

“repeat_selected_flag” indicates whether to repeatedly continue theanimation display for the button i in the selected state. Ifstart_object_id_selected and end_object_id_selected have the same value,this field will be set as 00.

“start_object_id activated” is such a field that, when rendering thebutton i in the active state by animation, the first number among theserial numbers that are assigned to the plurality of ODSs constitutingthe animation is described in this start_object_id_activated.

“end_object_id_activated is such a field that, when rendering the buttonin the active state by animation, the last number among the serialnumbers (i.e. object_id) that are assigned to the plurality of ODSsconstituting the animation is described in this end_object_id_activated.

Next, the button command is explained.

A button command (button_command) is a command to be executed when thebutton i is set in active state.

As follows, an example of interactive control by means of ICS isexplained. This example assumes ODS and ICS as illustrated in FIG. 44.FIG. 44 shows a relationship between the ODS included in the DSn and theICS. This DSn is assumed to contain ODS11-19, 21-29, 31-39, and 41-49.Among these ODSs, ODS11-19 respectively illustrate states of the buttonA; ODS21-29 illustrate states of the button B; ODS31-39 states of buttonC; and ODS41-49 states of button D. (Please refer to the parenthesis“}”) Then it is assumed that the state controls for these buttonA-button D are described in the button_infos (1), (2), (3), (4),respectively.

The executing timing of control by means of ICS coincides with thedisplay timing of arbitrary picture data pt1 in the moving picture ofFIG. 45, this means that an interactive screen tm1 made up of the buttonA-the button D will be displayed by being composed (gs1) into thispicture data pt1 (gs2). Since an interactive screen made of a pluralityof buttons in accordance with the contents of the moving picture, itbecomes possible to render very real images with use of buttons and bymeans of ICS.

FIG. 46 illustrates a description example of ICS, in a case where thestate of the button A-the button D is changed as shown in FIG. 47. Thearrows hh1 and hh2 in FIG. 47 symbolically represent the state changegenerated by neighbor_info( ) of button info(1). The neighbor_info( ) ofbutton info(1) has lower_button number to which the button C is set.Therefore if UO of MOVEDown key being pushed is generated while thebutton A is in a selected state (FIG. 47, up1), the button C will be ina selected state (FIG. 47, sj1). Since right_button_number inneighbor_info( ) of button info(1) is set as button B, if the UO ofMOVERight key being pushed is generated while the button A is in aselected state (FIG. 47, up2), the button B will be in a selected state(FIG. 47, sj2).

The arrow hh3 in FIG. 47 shows control for state change of buttoninfo(3) due to neighbor_info( ). Since upper_button_number inneighbor_info of button info(3) is set as button A, if UO of MOVEUp keybeing pushed is generated while the button C is in a selected state, thebutton A will return to be in a selected state.

Next, the images for the button A—the button D are explained. Here,assumption is made that the ODS11, 21, 31, and 41 have imagesillustrated in FIG. 49, and the like. Since in ICS, normal_state_info( )of button_info(1) has start_object_id_normal, end_object_id_normal thatindicate ODS11-13, the normal state for the button A is represented asanimation of ODS11-13. In addition, since the selected_state_info( ) ofbutton_info(1) has start_ovject_id_selected, end_object_id_selected thatindicate ODS14-16, the selected state of the button A is represented asODS14-16. As a result of this button A being set to be in selected stateby a user, the figure which is the image of the button A will change tothat of ODS14-16, from that of ODS11-13. Here, if arrangement is made inwhich repeat_normal_flag, repeat_select_flag are set to 1, in thenormal_state_info( ), selected_state_info( ) respectively, then theanimation of ODS11-13 and the animation of ODS14-16 will be displayedrepeatedly and continuously, as shown by “→(A)”, “(A)→”, “→(B)”, and“(B)→” of the drawing.

If plural ODSs, which are able to render animation, are assigned to thebutton A—the button D, and corresponding controls are described in ICS,button state control can be realized more subtly and quickly (e.g.changing the expression of the character of the image as the useroperation changes).

Next, the order of ODS in Display Set is explained. As aforementioned,ODS belonging to Display Set is indicated by ICS to represent one stateof a button. The order of ODS in Display Set is determined, inaccordance with the indication as to which state of a button it shouldrepresent.

More specifically, ODSs in Display Set are grouped under which statethey represent, (1) as representing normal state, (2) as representingselected state, (3) as representing active state, and so on. Each ofsuch groups, which represents one state of a button, is called“button-state group”. Then, these button-state groups are arranged inthe order such as “normal state→selected state→active state”. Definingthe order of ODS will accordingly define the order of the ODS in theDisplay Set.

FIG. 50 illustrates the order of ODS belonging to Display Set. In thesecond row of this diagram, three button-state groups of the Display Setare shown. This drawing shows three sets of ODS: an ODS set that rendersnormal state (ODS for Normal state); an ODS set that renders selectedstate (ODS for Selected state); and an ODS set that renders active state(ODS for Actioned state). These button-state groups are arranged in theorder of “normal state→selected state→active state”. The purpose of thisorder is to read the components that constitute the initial display ofthe interactive screen, ahead of reading of the other components thatconstitute the screen display after update.

The first row of FIG. 50 shows the Graphics Objects “An, Bn, Cn, Dn, As,Bs, Cs, Ds, Aa, Ba, Ca, Da”. The subscript “n” assigned to An, Bn, Cn,Dn represents normal state of a corresponding button. In the samemanner, the subscript “s” of As, Bs, Cs, Ds represents selected state ofa corresponding button, and the subscript “a” represents active state ofa corresponding button. The second row of FIG. 50 shows button-stategroups to which the Graphics Objects of the first row belong. It shouldbe noted, in this drawing, every set of ODS1-ODSn is assigned the samenumber, such as 1 and n. However, the sets are different from oneanother, and belong to N-ODSs, S-ODSs, and A-ODSs, respectively. Thisalso applies to every similar drawing hereinafter.

FIG. 51 illustrates state transition of an interactive screen in whichthe button-state groups of FIG. 50 are disposed.

The interactive screen of this drawing has plural states that are“initial display”, “update display according to 1^(st) user action”, and“update display according to 2^(nd) user action”. The arrows in thisdrawing represent user actions that trigger the corresponding statechange. According to this drawing, the four buttons A, B, C, and D have“normal state”, “selected state”, “active state”, respectively. It canbe understood that, so as to perform initial display, the GraphicsObject for rendering three normal states and the Graphics Object forrendering one selected state are necessary.

Even in a case where the default selected button is not defined andwhich button to be set in selected state will change dynamically,initial display will be realized once the decoding for the GraphicsObjects representing the normal state and the selected state, for eachbutton, are complete. Taking this into consideration, the presentembodiment disposes the button-state groups each corresponding to adifferent one of the states in the order of “normal state→selectedstate→active state”, as shown in the second row of FIG. 50. Suchdisposition realizes initial display even when the reading and decodingof ODSs constituting the active state are incomplete, and helps shortenthe period starting with reading start of Display Set and ending withcompletion of initial display.

Next, the following explains in which order the ODSs illustrated inFIGS. 48 and 49 are disposed. FIG. 52 illustrates the order of ODS inDisplay Set. In this drawing, ODSs for Normal state are constituted byODS11-13, ODS21-23, ODS31-33, and ODS41-43. The ODSs for Selected stateare constituted by ODS14-16, ODS24-26, ODS34-36, and ODS44-46, and theODSs for Actioned state are constituted by ODS17-19, ODS27-29, ODS37-39,and ODS47-49. ODS11-13 are for rendering the change in expression of thecharacter that is shown in FIG. 49. The same thing applies to ODS21-23,ODS31-33, and ODS41-43. Therefore by disposing these ODSs in the topbutton-state group, it become's possible to arrange the preparation ofinitial display even in the middle of reading out Display Set. Accordingto this, the interactive screen that takes in animation can be performedwithout delay.

Next, the order of ODSs that are multiple-referred by a plurality ofbuttons is explained. Here, multiple-referring means that object_id ofan ODS is indicated by two or more of normal_state_info,selected_state_info, and activated_state_info. By adopting suchmultiple-referring method, the selected state of a button can berendered using the Graphics Object for rendering the normal state of adifferent button. This enables sharing of the image of the GraphicsObject. Such sharing helps reduce the number of ODSs. In this case, aproblem concerning the ODS used in multiple-referring is whichbutton-state group this ODS belongs to.

More specifically, when the normal state of a button and the selectedstate of another button are rendered by one ODS, the thing to beconsidered is whether this ODS belongs to a button-state groupcorresponding to the normal state or to a button-state groupcorresponding to the selected state.

In this case, the ODS is disposed only once in the button-state groupthat corresponds to the state that appears earliest.

If a certain ODS is used in multiple-referring in both of the normalstate and in the selected state, this ODS will be disposed in thebutton-state group corresponding to the normal state (N-ODSs), and notin the button-state group corresponding to the selected state (S-ODSs).In addition, if another ODS is used in multiple-referring in both of theselected state and in the active state, this ODS will be disposed in thebutton-state group corresponding to the selected state (S-ODSs), and notin the button-state group corresponding to the active state (A-ODSs). Insum, ODSs in such a multiple-referring method will be disposed only oncewithin the button-state group corresponding to the earliest appearingstate.

Next, the order of ODSs in S-ODSs is explained. In S-ODSs, which ODScomes first depends on whether the default selected button is determinedstatically, or it is dynamic. The default selected button which isstatically determined has a valid value that is set (excluding 00) indefault_selected_button_number in ICS, and this value specifies thebutton. When default_selected_button_number indicates a valid value, andthat there is no ODS representing the default selected button in N-ODSs,the ODS representing the default selected button will be disposed first.

If default_selected_button_number indicates the value 00, the button setin the selected state as default will dynamically change according tothe state of the reproduction apparatus side.

A case where default_selected_button_number should be set to indicatethe value 0 is, for example, where AVClip to which Display Set has beenmultiplexed works as a joint point for a plurality of reproductionpaths. If for example, the preceding plural playback paths arerespectively first, second and third chapters, and the Display Set beingthe joint point is for displaying the buttons corresponding to thefirst, second, and third chapters, it is inadequate to decide, asdefault, the button to be in selected state indefault_selected_button_number.

In such a case, it is ideal to change the button to be in selected stateaccording to which one of the preceding plural reproduction paths iscurrently gone through, until reaching this Display Set (e.g. thesecond-chapter button when reaching from the first chapter, thethird-chapter button when reaching from the second chapter, and thefourth-chapter button when reaching from the third button). In a casewhere the button to be in selected state will change,default_selected_button_number will be designed to be invalid, i.e. thevalue 0 is set thereto. Since the button to be in selected state willchange, an arrangement of disposing a certain ODS at the beginning ofthe button-state group is not necessary.

FIG. 53 shows difference in arrangement of ODS in S-ODSs between a casewhere default_selected button_number is “=0”, and a case where it is“=button B”. In this drawing, the broken line ss1 shows the dispositionof ODSs in S-ODSs, in a case where default_selected_button_numberindicates the button B; and the broken line ss2 shows the disposition ofODSs in S-ODSs, in a case where default selected button_number indicatesthe value 0. As the notation of this drawing shows, whendefault_selected_button_number indicates the button B, the ODSBsindicating the button B's selected state is disposed first of S-ODSs,and the other buttons' ODSs are disposed thereafter. On the contrary,when default_selected_button_number indicates the value 0, the ODSAsindicating the button A's selected state is disposed first. As such,whether default_selected_button_number is valid or not causes the orderof S-ODSs to change greatly.

Next, the following explains how Display Set having these ICS and ODS isdisposed on the reproduction timeline for AVClip. DTS and PTS of ODS canbe set based on the expressions shown in the first embodiment. On thecontrary, DTS and PTS of ICS will be different from those shown in thefirst embodiment. The following explains the values for DTS and PTS ofICS.

When immediately after Epoch start, PTS in ICS will be set as an equalto or greater value than the value resulting from summing (1) PTS valueof the ODS whose decode time is the last among the ODSs constituting theinitial display of DSn, (2) time required to clear the Graphics Plane,and (3) write time for writing the Graphics Object obtained by ODSdecoding to the Graphics Plane. On the other hand, when at AcquisitionPoint, it will be set to be equal to or grater than the value obtainedby adding (3) the plane write period to (1) PTS value of ODS.

When default_selected_button_number is indicated in ICS, the initialdisplay can be performed as long as i) ODS decoding for rendering thenormal state of all the buttons and ii) ODS decoding for rendering theselected state of the default button. The ODSs for rendering theselected state of plural buttons at the initial display are calledS-ODSs, and the ODS whose decoding time comes first among the ODSs (inthis case, the ODS for rendering the default button) is calledS-ODSsfirst. The PTS value of this S-ODSsfirst is set as the PTS valueof the ODS whose decoding time comes last, and is used as a referencevalue of PTS in ICS.

When default_selected_button_number is not indicated in ICS, any buttoncan be in selected state. Therefore preparation for the initial displaywill not be complete until preparation for rendering the normal stateand the selected state for all the buttons. Among the S-ODSs forrendering the selected state of the plural buttons at the initialdisplay, the one whose decoding time comes last is called S-ODSslast.The PTS value for this S-ODSslast is set as the PTS value of the ODSwhose decoding time comes last, and is used as a reference value of PTSin ICS.

If the ending time for decoding the S-ODSsfirst is assumed to bePTS(DSn[S-ODSsfirst]), PTS(DSn[ICS]) will be a value resulting fromadding, to PTS (DSn[S-ODSsfirst]), (2) the time required for clearingthe Graphics Plane, and (3) write time for writing the Graphics Objectobtained from ODS decoding to the Graphics Plane.

Assume here that, within the Graphics Plane, the width and the height ofa rectangular area for rendering picture are respectively defined as“video-width” and “video-height”, and that the rate of writing to theGraphics Plane is 128 Mbps. Then the time required for clearing theGraphics Plane is expressed as“8*video_width*video_height//128,000,000”. When this is expressed intime accuracy of 90 KHz, the clearing time(2) of the Graphics Plane willbe 90,000*(8*video_width*video_height//128,000,000).

In addition, assume here that the total size of the Graphics Objectsspecified by all the button information included in ICS isΣSIZE(DSn[ICS.BUTTON[i]]), and that the rate of writing to the GraphicsPlane is 128 Mbps, then the time required for writing to the GraphicsPlane is expressed as ΣSIZE(DSn[ICS.BUTTON[i]])//128.000,000. If this isexpressed in time accuracy of 90 KHz, the clearing time (2) of theGraphics Plane is 90,000*(ΣSIZE(DSn[ICS.BUTTON[i]]))//128.000,000).

Here, ΣSIZE(DSn[ICS.BUTTON[i]]) is the total size of thefirstly-displayed Graphics Object, among the Graphics Objectsrepresenting all the buttons. This ΣSIZE(DSn[ICS.BUTTON[i]] will yield adifferent value, in a case where the default selected button has alreadydetermined, from in a case where the default selected button changesdynamically. When the default selected button has been staticallydetermined, ΣSIZE(DSn[ICS.BUTTON[i]]) will be a total of 1) thefirstly-displayed ODS among the plural ODSs for representing theselected state of the default selected button, and 2) thefirstly-displayed ODS among the plural ODSs for representing the normalstate of the buttons except the default selected button.

On the contrary, when the default selected button changes dynamically,it should assume the case where the write time is the longest, becauseit is hard to know which button will be the default selected button. Inthis case, it is considered that the Graphics Object to be firstlydisplayed is the Graphics Object having the largest size(Max(ODSn1.ODSs1) among 1) the Graphics Objects representing the firstpage in the normal state of an arbitrary button×(ODSn1), and 2) theGraphics Objects representing the first page in the selected state ofthe button×(ODSs1).

The result of summing this Max(ODSn1,ODSs1) of reach button will beΣSIZE(DSn[ICS.BUTTON[i]]).

FIGS. 54A, 54B illustrate which values ΣSIZE(DSn[ICS.BUTTON[i]]) takes,in a case where N-ODSs includes plural ODSs constituting the buttonsA-D, and where S-ODSs includes plural ODSs constituting the buttons A-D.Here, when default_selected_button_number indicates a valid value,ΣSIZE(DSn[ICS.BUTTON[i]]) will be the total size for the four ODSs shownby the frame in thick line. “As1” is the ODS firstly displayed among theplural ODSs representing the selected state of the button A. “Bn1”,“Cn1”, and “Dn1” represent the corresponding ODSs firstly displayedamong the plural ODSs representing the normal states of the button B-thebutton D. When these sizes are expressed in size( ),ΣSIZE(DSn[ICS.BUTTON[i]]) will be:size(As1)+size(Bn1)+size(Cn1)+size(Dn1).

On the other hand, when default_selected_button_number is “=0”,ΣSIZE(DSn[ICS.BUTTON[i]]) will be:

Greater ODS of An1,As1+Greater ODS of Bn1,Bs1+Greater ODS ofCn1,Cs1,+Greater ODS of Dn2,Ds1.

Therefore ΣSIZE(DSn[ICS.BUTTON[i]]) is expressed as follows.ΣSIZE(DSn[ICS.BUTTON[i]])=max(size(Cn1),size(Cs1))+max(size(Dn1),size(Ds1))

Using the above-stated expression, PTS(DSn[ICS]) immediately after thestarting of Epoch Start is expressed as follows.PTS(DSn[ICS]) ≥ PTS(DSn[S − ODSsfirst]) + 90,000 * (8 * video_width * video_height//128,000,000) + 90,000 * (∑SIZE(DSn[ICS.BUTTON[i]])//128,000,000)

An example of realizing sync display, by setting PTS and DTS as above,is shown in FIG. 55. This drawing assumes a case where the button isdisplayed at the display timing of any picture data py1 in the movingpicture. In such a case, the PTS value in ICS should be set so as tocoincide with the display time-point of the corresponding picture data.

In addition, the PTS value in ODS should be set at the time-point (1) ofthis drawing, because, by the time obtained by subtracting, from the PTSin ICS, the clearing period of the screen “cd1” and the transfer periodof the Graphics Object “td1”, decoding of the ODS whose decoding timecomes last among the ODSs that constitute the initial display of DSn,should be complete. Furthermore, because decoding of ODS require s theperiod of dd1, the DTS value of ODS should be set prior to this PTS bythe period of dd1.

FIG. 55 has only one ODS to be combined with a moving picture, which isa simplified example. In order to realize the initial display of theinteractive screen to be combined with the moving picture among theplural ODSs, the PTS and DTS in ICS, and PTS, DTS in ODS should be setas shown in FIG. 56.

FIG. 56 shows how DTS and PTS are set in a case where the initialdisplay of the interactive screen is constituted by plural ODSs, andwhere the default selected button is statically determined. If decodingfor the S-ODSsfirst whose decoding is performed the last, among the ODSsfor realizing the initial display, will end during the period dd1 ofthis drawing, PTS(DSn[S-ODSsfirst]) of this S-ODSsfirst should be set toindicate the time of the period dd1.

Furthermore, before initial display, screen clearing and transferring ofalready decoded Graphics Objects should be performed. ThereforePTS(DSn[ICS] of ICS should be set to be after the time obtained byadding, to the value of this PTS(DSn[S-ODSsfirst]), the period requiredfor screen clearing (90,000*(8*video_width*video_height//128,000,000))and the transfer period of the decoded Graphics Object(90,000*(ΣSIZE(DSn[ICS.BUTTON[i]])//128,000,000)).

FIG. 57 shows how DTS and PTS are set in a case where the initialdisplay of the interactive screen is constituted by plural ODSs, andwhere the default selected button is not determined. If decoding for theS-ODSslast whose decoding is performed the last, among the ODSs forrealizing the initial display, will end during the period dd2 of thisdrawing, PTS(DSn[S-ODSslast]) of this S-ODSslast should be set toindicate the time of the period dd2.

Furthermore, before initial display, screen clearing and transferring ofalready decoded Graphics Objects should be performed. ThereforePTS(DSn[ICS] of ICS should be set to be after the time obtained byadding, to the value of this PTS (DSn[S-ODSslast]), the period requiredfor screen clearing (90,000*(8*video_width*video_height//128,000,000))and the transfer period of the decoded Graphics Object(90,000*(ΣSIZE(DSn[ICS.BUTTON[i]])//128,000,000)).

It should be noted here that the sync control by means of PTS in ICS,stated above, includes not only control to display the button at acertain timing on the reproduction timeline, but also include thecontrol to enable the display of Popup menu during a certain period onthe reproduction timeline. The Popup menu is a menu displayed, as apopup, by a push directed to the menu key provided for the remotecontroller 400. The sync control by means of PTS in ICS also includesenabling this Popup display at the display timing of certain picturedata in the AVClip. The ODSs constituting this Popup menu are decodedfirst and then written to the Graphics Plane, just as the ODSsconstituting a button. Unless the writing to the Graphics Plane has beencomplete, it is impossible to answer the menu call by a user. In view ofthis, The time at which the Popup display becomes possible is written toPTS in ICS, at the sync display of the Popup menu.

Following the explanation on the recording medium of the presentinvention stated above, a reproduction apparatus according to thepresent invention is explained as follows. The internal structure of thereproduction apparatus according to the second embodiment issubstantially the same as that of the first embodiment, except someimprovements for the Object Buffer 15, and for the Graphics controller17. Therefore the improvements for the Object Buffer 15, and for theGraphics controller 17 are detailed as follows.

Graphics objects that are obtained by the decoding performed by theStream Graphics processor 14 and that are for constituting aninteractive screen are disposed in the Object Buffer 15 according to thesecond embodiment. FIG. 58 illustrates the content of the Object Buffer15 in comparison with the Graphics Plane 8. The content of the ObjectBuffer 15 assumes a case in which the ODS shown in FIG. 48 and FIG. 49are written to the Object Buffer 15. The examples of FIG. 48 and FIG. 49realizes the four-button animation by 36 ODS (ODS11-ODS49), where theODS representing all the frames of this animation are stored in thisObject Buffer 15, and the display position of each of the ODSs stored inthis Object Buffer 15 is defined in the Graphics Plane 8. This displayposition is defined by Button_horizontal_position andButton_vertical_position of the corresponding button information.Animation is realized by writing the plural ODSs stored in the ObjectBuffer 15 to a corresponding display position of the Graphics Plane 8,by transferring one frame at a time.

The Graphics controller 17 of the second embodiment interprets the ICSdisposed unit the Composition buffer 16, and executes control based onthe ICS The execution timing of this control is based on the value ofPTS assigned to the ICS. The important task of this Graphics controller17 is a writing operation at the initial display time of the interactivescreen, and at the update time. The following describes, with referenceto FIG. 59, the writing operation at the initial display time of theinteractive screen and at the update time. FIG. 59 illustrates theoperation performed by the Graphics controller at the initial displaytime. As this diagram shows, the Graphics controller 17 performs controlso that the ODSs belonging to S-ODSs of the button A are written to thedisplay position defined by Button_horizontal_position andButton_vertical_position of the button information at the button A; andsimilarly, the ODSs belonging to the N-ODSs of the buttons B, C, D arewritten to the respective display positions defined by the correspondingButton_horizontal_position and Button_vertical_position of the buttoninformation of the buttons B, C, and D. Note here that the arrows w1,w2, w3, and w4 symbolically show the aforementioned writing. Byperforming the writing, the initial display shown in FIG. 51 will beperformed. What should be noted here is that not all the ODSs arenecessary for realizing the initial display of the interactive screen,and as long as the Object Buffer 15 contains the ODSs belonging toS-ODSs of the default selected button and the ODSs belonging to N-ODSsof the other buttons, it is enough for completing the initial display ofthe interactive screen. Therefore when the ODSs belonging to S-ODSs ofthe default selected button and the ODSs belonging to N-ODSs of theother buttons have been decoded, it can be said to be ready for theGraphics controller 17 to start performing writing for the initialdisplay of the interactive screen.

FIG. 60 illustrates the operation performed by the Graphics controller17 when the interactive-screen update is performed according to1stUserAction(MoveRight). As this drawing shows, the Graphics controller17 performs control so that the ODS belonging to S-ODSs of the button Bis written to the display position defined by Button_horizontal_positionand Button_vertical_position of the button information at the button B;and similarly, the ODSs belonging to the N-ODSs of the button A arewritten to the display position defined by Button_horizontal_positionand Button_vertical_position of the button information of the button A.Note here that the arrows w5, w6, w7, and w8 symbolically show theaforementioned writing. By performing the writing, the state changeshown in FIG. 51 will be realized. The buttons C and D are in the normalstate, just as in the initial display time, but the writing to theGraphics Plane 8 is being continually performed therefor, so as tocontinue the animation.

In the similar manner as above, FIGS. 61 and 62 illustrate theoperations performed by the Graphics controller 17, at the interactivescreen update when the 1stUserAction is “MoveDown” and “Activated”. Atthe interactive screen update time, The S-ODSs and the A-ODSs of thebuttons other than the default selected button, and so it is desiredthat all the ODSs have been stored in the Object Buffer 15.

In the reproduction apparatus, constructed as in the above, eachcomponent performs decoding operations in pipeline processing method,Just as in the first embodiment.

FIG. 63 is a timing chart illustrating a pipeline processing performedby the reproduction apparatus. The fourth row shows the Display Set ofBD-ROM, and the third row shows reading periods of ICS, PDS, ODS to theCoded Data buffer 13. The second row shows decode periods of ODSs, wherethe decoding is performed by the Stream Graphics processor 14. The firstrow shows operation periods of the Graphics controller 17. The decodingstart time for the ODSs are shown by DTS11, DTS12, and DTS 13,respectively. The storing, to the Coded Data buffer 13, of the first ODS(N-ODSs [ODS1]) among those belonging to N-ODSs will be complete by theDTS11. The storing, to the Coded Data buffer 13, of the last ODS (N=ODSs[ODSn]) among those belonging to the N-ODSs, will be complete by theDTS12. Asu such, each ODS will have been read to the Coded Data buffer13 by the time shown by its own DTS.

On the other hand, the decode end time of each ODS is shown by PTS11,PTS12, and PTS13 of the drawing. The decoding of N-ODSs(ODS1) performedby the Stream Graphics processor 14 will be complete by PTS11; and thedecoding of N-ODSs (ODSn) will be complete by PTS12. As such, by thetime shown by the DTS of each ODS, the ODS has been read to the CodedData buffer 13, and each read ODS to the Coded Data buffer 13 will bedecoded and written to the Object Buffer 15 by the time shown by the PTSshown by the corresponding PTS. The Stream Graphics processor 14performs these series of operation, in pipeline processing method.

When the default selected button is determined statically, all theGraphics Objects necessary for the initial display of the interactivescreen will be ready on the Object Buffer 15 when the decoding iscomplete for 1) the button-state group corresponding to the normalstate, and 2) the first ODS of the button-state group corresponding tothe selected state. In this drawing, at the time shown by PTS13, all theGraphics Objects necessary for the initial display of the interactivescreen get ready.

In this drawing, the period cd1 in the first row is the period necessaryfor clearing the Graphics Plane 8. In addition, the period td1 is theperiod necessary for writing, to the Graphics Plane 8, the GraphicsObjects constituting the first page of the interactive screen, which areamong the Graphics Objects obtained on the Object Buffer 15. The exactstoring place of the Graphics Objects in the Graphics Plane 8 is theplace shown by button_horizontal_position and button_vertical_position.In other words, cd1 (screen clearing period) td1 (writing period of theGraphics Object having been decoded) are added to PTS13 of ODS,decompressed graphics constituting the interactive screen will beobtained on the Graphics Plane 8 in the obtained period. Then, by 1)making the CULT unit 9 perform a color conversion of decompressedgraphics, and 2) making the adding unit 10 combine the decompressedpicture stored in the video plane 6, the composition image will beobtained.

As opposed to the case where the initial display is performed afterdecoding all the ODSs included in the Display Set, it becomes possible,in the above-stated case, to perform an initial display regardless ofwhether decoding of a button-state group corresponding to the selectedstate has been complete, or whether decoding of a button-state groupcorresponding to the active state has been complete. Therefore theinitial display will be performed earlier in this case by the period hy1in the drawing.

It should be noted, in this drawing, every set of ODS1-ODSn is as signedthe same number, such as 1 and n. However, the sets are different fromone another, and belong to N-ODSs, S-ODSs, and A-ODSS, respectively.This also applies to every similar drawing hereinafter.

In the graphics decoder 12, even while the Grphics controller 17continues executing clearing of the Graphics Plane 8 or writing to theGraphics Plane 8, the Stream Graphics processor 14 continues performingdecoding (decode period of ODSn, decode period of ODS1, and decodeperiod n of ODSn, in the second row). Therefore, it becomes possible tofinish decoding of the other ODSs than those being treated by theGraphics controller 17, earlier than conventionally, since the otherODSs will be decoded simultaneously with the decoding of ODSs beingtreated by the Graphics controller 17. Since it becomes possible to havepreparation for updating the interactive screen early by completing thedecoding of the other ODSs, the interactive screen update, which willuse the other ODSs, will be accordingly finish earlier thanconventionally. The aforementioned pipeline processing enables both theinitial display of the interactive screen and the updating thereof, tobe performed without delay.

FIG. 63 assumes the case where the default selected button has beenstatically determined. On the contrary, FIG. 64 is a timing chartillustrating a pipeline processing by the reproduction apparatus in acase where the default selected button changes dynamically. When thedefault selected button changes dynamically, the Graphics Objectsnecessary for the initial display will be ready, when all the ODSsbelonging to the button-state group have been decoded and the GraphicsObjects are obtained in the Graphics Plane. As opposed to the case wherethe initial display is performed after decoding all the ODSs included inthe button-state group corresponding to the active state, theabove-stated case enables the initial display to be performed regardlessof whether decoding of the button-state group corresponding to theactive state has been complete. Therefore, the initial display will beperformed earlier in this case by the period hy2 in the drawing.

FIG. 65 is a timing chart illustrating chronological transitions inoccupancy of the Graphics Plane 8, the Object Buffer 15, the Coded Databuffer 13, and the Composition buffer 16. The notations of occupancyused in this drawing conform to those used in FIG. 30. Since the ODSsconstituting N-ODSs, S-ODSs, A-ODSs are to be decoded in the secondembodiment, the number of simply increasing parts and of simplydecreasing parts is more than in FIG. 30. Except for this difference,FIG. 65 is the same as FIG. 30. Just as in the first embodiment, thegraphs for example in FIG. 65 are illustrated using: DTS and PTSassigned to ODS; DTS and PTS assigned to ICS; size and transfer rate ofeach buffer illustrated in FIG. 27. Furthermore, by creating suchgraphs, users can know how the state of each buffer changes, at theauthoring stage. Since the transition of the state of each buffer can beadjusted by updating DTS and PTS, it becomes possible, also in thisembodiment, to avoid generation of decoding load which would exceed thespecification of the decoder at the side of the reproduction apparatus,and to avoid buffer overflow incident to reproduction. According tothis, the hardware/software implementation will become easy, in thedeveloping phase of reproduction apparatuses.

Next, software improvement necessary for realizing the reproductionapparatus of the second embodiment is explained.

FIG. 66 is a flowchart showing the process of loading operation offunctional segment. This drawing is dawn based on the flowchart of FIG.31. The difference is that, after Step S29, Steps S36 and S67 are addedto FIG. 66.

Step S36 is to judge whether command_update_flag is 1. If it is 1 (StepS36:Yes), only button command in the button information is loaded to theCoded Data buffer 13, and the others are ignored (Step S37). If it is 0,the control is moved to Step S22, thereby ignoring the ICS representingAcquisition Point (Step S24).

Next, assuming the case where multiplexing is performed as in FIG. 67,the following explains how DS is read out. The example of FIG. 67multiplexes three DS with a moving picture. Among the three DS, thefirst DS1 has Epoch_Start as Composition_state, includes a buttoncommand called LinkPL(PL#5), and whose Command_update_flag is set as 0.

DS10 is “Duplicate” of DS1, and has Acquisition Point asComposition_state, includes a button command called LinkPL(PL#5), andwhose Command_update_flag is set as 0.

DS20 is “Inherit” of DS1, and has Acquisition Point asComposition_state. The difference from DS1 is the button command(LinkPL(PL#10)), and so as to representing this, its Command_update_flagis set as 1.

Assume here that these three DS and the moving picture are multiplexedin an AVClip, and a skip operation to a picture data pt10 of ms1 isperformed. In this case, the DS10 which is closest to a skipping targetis the target of FIG. 66. At Step S27, composition_state will be judgedto be Acquisition Point, but no preceding DS exist in the graphicsdecoder 12. Therefore the ignore flag is set as 0, and this DS10 isloaded to the Coded Data buffer 13 of the reproduction apparatus (hs1 ofFIG. 68). On the other hand, when the target of a skip operation fallsafter the position at which Display Set exists (ms2), Display Set 20(hs2 of FIG. 68) will be read to the Coded Data buffer 13.

FIG. 70 shows loading of DS1, DS10, and DS20, in a normal reproductionas performed in FIG. 69. Among the three DS, DS1 whose Composition_stateof ICS is Epoch Start is loaded to the Coded Data buffer 13, as it is(Step S23). However, DS10, whose Composition_state of ICS is AcquisitionPoint, has the ignore flag of 1 (Step S29). Therefore the functionalsegments constituting DS10 will not be loaded to the Coded Data buffer13, and will be ignored instead (Step S24). Furthermore, as for DS20, itis true that its Composition_state of ICS is Acquisition Point, but itsCommand_update_flag is set as 1. Therefore Step S36 yields “Yes”, and soonly button command thereof is loaded, and only the button command ofICS of DS on the Coded Data buffer 13 is replaced with the buttoncommand of DS20 (Step S37). However, the ignore flag still represents 1,and so the others different from this button command will not be loaded,and ignored instead.

When reaching DS20, the display content stays the same, however thebutton command has changed from LinkPL(#5) of DS, to LinkPL(#19). Suchreplacement of button command enables control of changing the contentsof a button command. Next, the processing performed by the Graphicscontroller is explained. FIG. 71 is a flowchart illustrating the mainroutine of the processing performed by the Graphics controller 17. Inthis flowchart, the following three operations are repeatedly executed:time stamp sync operation (Step S35), animation display operation (StepS36), and UP operation (Step S37).

Here, the processing performed by the Graphics controller 17 isexplained. The processing performed by the Graphics controller 17 isgreatly changed from what are shown in FIGS. 36-38, to what are shown inFIGS. 71-78. FIG. 71 is a flowchart illustrating the main routine of theprocessing performed by the Graphics controller 17. FIG. 72 is aflowchart illustrating the processing for realizing sync control thatuses time stamp. In this flowchart, judgment is performed as to any ofthe conditions of Steps S41, S43-S47 holds. If any of the conditionsholds, a corresponding operation is performed, and then goes back to themain routine. The stated processing is a subroutine.

Step S41 is to judge whether the current reproducing point is one of thetime represented by the PTS of the S-ODSsfirst, and the time representedby the PTS of the S-ODSslast. If the current reproducing point is judgedto be one of the stated times, the period α thereof is calculated. Theperiod α is obtained by summation of (2) period required for clearing aGraphics Plane, and (1) period required for writing the Graphics Objectobtained by ODS decoding, to the Graphics Plane.

At Step S42, the Graphics controller 17 refers to Composition_state inICS, and a) if Composition_state is Epoch Start, sets a to be “planeclearing period (2)+plane writing period (3)”; b) if Composition_stateis Acquisition Point, sets a to be the plane writing period (3).Calculation of the plane writing period (3) is performed as follows: ifdefault_selected_button_number is a valid value, the calculating methodof FIG. 54A is used; and if default_selected_button_number is 0, thecalculating method of FIG. 54B is used. When α is calculated, thecontrol will return to a loop processing.

Step S43 is to judge whether the current reproducing point is the timerepresented by PTS-α in ICS. If the judgment results in affirmative, awriting operation to the Graphics Plane 8 is performed (Step S51), andthe control returns to the main routine.

Step S45 is to judge whether the current reproducing point is PTS inICS. If the judgment results in affirmative, then the outputting of thecontents of the Graphics Plane 8 is instructed. The destination of thecontents is the CLUT unit 9. The CLUT unit 9 performs color conversionto the contents. Then the interactive screen will be combined with thecontents of the video plane 9. As a result, the initial display isperformed (Step S52). Then, the variable “animation (p) (p=1, 2, 3 . . .n) is set to be 0 (Step S53), and the control returns to the mainroutine. Here, the variable animation (p) is a global variable thatindicates which number of frame in the frame sequence is currentlydisplayed, used in executing the animation display of the button (p) (aglobal variable is a variable that is valid throughout a plurality offlowcharts). Therefore at Step S53, button (p) of all the buttons willbe set to be 0.

Step S46 and Step S47 are to judge whether the current reproducing pointhas reached the time information described in ICS.

Step S46 is to judge whether the current reproducing time is the timerepresented by selection_TimeOut_PTS, and if the judgment results inaffirmative, an operation for activating the button represented bydefault_activated_button_number is performed, and the control returns tothe main routine (Step S54).

Step S47 is to judge whether the current reproducing point isComposition_TimeOut_PTS, and if the judgment results in the affirmative,the screen is cleared, then the control returns to the main routine(Step S55). In the sync operation stated above, each of the operationsat Step S51 and Step S54 are performed as a subroutine. Following this,the subroutine at Step S51 is explained with reference to FIG. 73.

FIG. 73 is a flowchart illustrating the operation of writing the initialdisplay of the menu, to the Graphics Plane 8. Step S64 is to judgewhether Composition_state in ICS is Epoch Start, and if the judgment isin the affirmative, the Graphics Plane is cleared at Step S65, and theoperations of Steps S66-S73 are performed. The period required forclearing the Graphics Plane 8 is the period cd1 in FIG. 56 and FIG. 57.If the judgment of Step S64 is in the negative, Step S65 is skipped, andthe operations of Steps S66-S73 are performed.

Steps S66-S73 form a loop processing, which is to be repeated for eachpiece of button information of ICS (Step S66, S67). Button informationthat should go through this loop processing is called button information(p).

Step S67 is to judge whether the indication bydefault_selected_button_number is valid or not. Step S68 is to judgewhether the button_info(p) is button information corresponding to thedefault selected button indicated by default_selected_button_number.

If the judgment in Step S68 is in the negative, the Graphics Object ofstart_object_id_normal, indicated by normal_state_info of button_info(p)is found from the Object Buffer 15, and is identified as Graphics Object(p) (Step S69).

If the judgment in Step S68 is in the affirmative, the Graphics Objectof start_object_id_selected, indicated by selected_state_info ofbutton_info(p) is found from the Object Buffer 15, and is identified asGraphics Object(p) (Step S70), then the button (p) is set as the currentbutton (Step S71). The current button is a button that has been set tobe in selected state in the interactive screen currently displayed. Thereproduction apparatus stores the identifier of this current button, asPSR(10).

Once the Graphics Object (p) has been identified as a result of Step S69and Step S70, the Graphics Object (p) is written to the position on theGraphics Plane 8, which is indicated by button_horizontal_position andbutton_vertical_position of button_info (p) (Step S72). By repeating theabove-described operations for each piece of button information, thefirst Graphics Object, which is among the plural graphic objects eachshowing the state of a corresponding button, is to be written to theGraphics Plane 8. The period required to perform the operation, which isdirected to the Graphics Object that is necessary for at least theinitial display of the Object Buffer 15, is shown by the period td1 ofFIG. 56 and FIG. 57.

When default_selected_button_number is “=0”, and that the defaultselected button changes dynamically, Step S67 will be No, and whetherthe button_info(p) corresponds to the current button is judged. If thejudgment of Step S67 results in the affirmative, the control goes toStep S70; and if the judgment results in negative, the control is movedto Step S69.

Next, the subroutine processing at Step S54 is explained, with referenceto FIG. 74.

FIG. 74 is a flowchart illustrating the automatic activation processingfor the default selected button. First, whetherdefault_activated_button_number is 0 or FF is judged (Step S75). If thejudgment of Step S75 results in “00”, no processing is performed and thecontrol returns to the main routine; and if the judgment of Step S75results in “FF”, the current button i is changed into active state (StepS77), the variable animation (i) is set as 0, and the control returns tothe main routine (Step S78).

If the judgment of Step S75 results in neither “00” nor “FF”, the buttonspecified as default_activated_button_number is set as the currentbutton (Step S76), the current button i is changed into active state(Step S77), the variable animation (i) corresponding to the currentbutton i is set as 0, and the control returns to the main routine (StepS78).

The above-stated processing enables the button in selected state, to bechanged into active state after a predetermined time.

Next, the animation by means of the menu (Step S36) is explained. FIG.75 is a flowchart showing the animation display processing.

Here, the initial display is realized by writing a Graphics Object tothe Graphics Plane 8, the Graphics Object having been specified by 1)start_object_id_normal of normal_state_info and 2)start_object_id_selected of selected_state_info, for each button_info.Here, “animation” is a processing to update the Graphics Plane with anarbitrary frame of each button (i.e. Graphics Object of qth frame),every time a cycle of the loop processing of Step S35-Step S37 iscompleted. This update is performed by returning the main routine, bywriting the Graphics Objects indicated by normal_state_info andselected_state_info of button_info, one by one to the Graphics Plane 8.Here, the variable q is used in identifying each Graphics Objectindicated by normal_state_info and selected_state_info of button_info,for each piece of button information.

The processing for realizing this animation display is detailed withreference to FIG. 75. This flowchart assumes a case whererepeat_normal_flag and repeat_selected_flag, of CIS, a reset to indicate“repetition necessary”, to simplify explanation.

Step S80 is to judge whether the initial display has finished. If thejudgment of Step S80 results in the negative, the control returnswithout performing any processing; if the judgment of Step S80 resultsin the affirmative, Step S81-Step S93 are performed. Step S81-Step S93constitute a loop processing of repeating the operations of StepS83-Step S93, for each button_info in ICS (Step S81, Step S82).

Step S83 is to set the variable animation(p) correspondingbutton_info(p), to the variable q. By performing this step, the variableq will indicate the current number of frames, which corresponds to thebutton_info(p).

Step S84 is to judge whether button_info (p) corresponds to the buttoncurrently in selected state (hereinafter “current button”).

If the button_info (p) is judged other than the current button, judgmentof Step S86 is performed.

Step S86 is to judge whether the current button is in active state, andif judging affirmatively, an identifier resulting from adding thevariable q to start_object_id actioned inbutton_info(p).actioned_state_info is set to be ID(q). Then, one buttoncommand among those included in button_info(p) is executed (Step S88).

If the current button is judged not be in active state, an identifierresulting from adding the variable q to start_object_id_selected inbutton_info (p) selected_state_info is set to be ID(q). (Step S89).

Once ID(q) is determined as a result of the above operations, theGraphics Object (p), having the ID(q) and existing in the Object Buffer15, is written to the position on the Graphics Plane 8 indicated bybutton_horizontal_position and button_vertical_position ofbutton_info(p) (Step S90).

By the above-explained loop processing, among the plural GraphicsObjects that constitute the selected state (or the active state) of thecurrent button, and the normal state of the other buttons, the GraphicsObject corresponding to qth page is written to the Graphics Plane 8.

Step S91 is to judge whether start_object_id_normal+q has reachedend_object_id_normal. If the judgment of Step S91 results in thenegative, the value resulting from incrementing the variable q by 1 isset as the variable “animation (p)” (Step S92). If the judgment of StepS91 results in the affirmative, the variable “animation (p)” isinitialized to have the value 0 (Step S93). The above-stated operationsare repeated to all the button_info in ICS (Step S81, Step S82). Whenall the button_info have undergone the above operations, the controlwill return to the main routine.

During the above-explained Step S80-Step S93, each time the main routine(Step S35-Step S37) is performed one time, the image of each button ofthe interactive screen will be updated to a new Graphics Object. Thismeans that, when the above-mentioned main routine (Step S35-Step S37) isperformed several times, a so-called animation is realized. In theanimation, the Graphics controller 17 adjust time so that the displayinterval for one frame of Graphics Object will be the value indicated byanimation_frame_rate_code.

It should be noted here that, at Step S88, the button commands includedin button_info (p) are executed one by one. However, it is also possibleto execute the button commands collectively, after the series ofGraphics Objects corresponding to the active state has been displayed.Next, the process of UO operation, which is to be performed at Step S37of the main routine, is explained with reference to FIG. 76.

FIG. 76 is a flow chart showing the process of UO operation. In thisflowchart, it is judged whether any of the conditions of Step S100-StepS103 holds. If any of the conditions holds, a corresponding processingis performed, and then goes back to the main routine. Step S100 is tojudge whether UomaskTable is set as “1”, and if the judgment is in theaffirmative, the control will return to the main routine, withoutperforming any processing.

Step S101 is to judge whether MoveUP/Down/Left/Right key has beenpushed. If the judgment is in the affirmative, the current button ischanged (Step S104), and it is then judged whether auto_action_flag ofthe current button is 01 (Step S108). If the judgment of Step S108 is inthe negative, the control returns to the main routine. If the judgmentof Step S108 is in the affirmative, the control is moved to Step S105.

Step S102 is to judge whether the activated key has been pushed. If thejudgment is in the affirmative, the current button i is changed to anactive state (step S105). Then, the variable “animation(i)” is set to be0 (Step S106).

Step S103 is to judge whether it is a case of numerical value inputting.If the judgment is in the affirmative, a corresponding numericalinputting operation is performed (Step S107), and the control returns tothe main routine. Among the processes in FIG. 76, each of Step S104 andStep S107 is a subroutine. The processes of these subroutines are shownin FIG. 77 and FIG. 78. The following is an explanation on theseflowcharts.

FIG. 77 is a flowchart showing the process of current button changingoperation. First, among upper_button_number, lower_button_number,Left_button_number, and right_button_number, which belong toneighbor_info of the current button, the one corresponding to the pushedkey is identified (Step S110).

Then the current button is set as “button i”, and the button to be thenew current button is set as “button j” (Step S111). Step S112 is tojudge whether the button j set in Step S111 corresponds to the button i.If they correspond to each other, the control will return to the mainroutine without performing any processing. If they do not correspond toeach other, the button j is set as the current button (Step S113), thevariable “animation(i)” and the variable “animation(j) are set to be 0,and the control returns to the main routine (Step S114).

FIG. 78 is a flowchart showing the process of numerical value inputtingoperation. Judgment is performed as to whether there is Button info.jthat has button_number matching the inputted numerical value (StepS121). Then judgment is performed as to whethernumerically_selectable_flag, of Button info.j, is 1 (Step S122). If StepS121 and Step S122 are “Yes”, the current button is changed to a normalstate, and the button j is set as the current button (Step S123), andthe variable “animation(i)” and the variable “animation(j) are set to be0 (Step S124). After these operations, judgment is performed as towhether auto_action_flag, of Button info.j, is 1 (Step S125). If thejudgment is in the negative, the control returns to the main routine.

If the judgment is in the affirmative, the current button is changed toan active state at Step S126, and the control returns to the mainroutine.

If any one of Steps S121-S122 is No, the control will return to the mainroutine.

The Graphics controller 17 performs the above processes, for performingsync display. Here, please note that if performing an interactive screendisplay triggered by a user operation using Popup display or the like,the Stream Graphics processor 14 and the Graphics controller 17 performthe following operations, which are the same operations performed forperforming sync display. By performing the following operations, theGraphics Object is obtained in the Graphics Plane 8. After the GraphicsObject is obtained as mentioned above, it is waited until the currentreproducing point passes the time indicated by the PTS assigned to ICS.Then, after the mentioned time, if the UO controller 18 receives UOindicating a menu call, it will be outputted to the CLUT unit 9, andinstructs the CULT unit 9 to perform the Graphics Object stored in theGraphics Plane 8. If such outputting is performed in syncronization withthe UO, a Popup display in accordance with the push of the menu callwill be realized.

The above explanation sets forth the setting of PTS in ICS; and DTS inODS and PTS, which belong to DSn. However, it does not mention DTS inICS; DTS and PTS in PDS; DTS and PTS in END. In view of this, thefollowing explains the time stamps relating to these. Since WDS does notexist in the second embodiment, ICS should be loaded to the Compositionbuffer 16, prior to 1) the decoding start time of the first PDS (PDS1)of DSn (i.e. DTS(DSn[ODS1])), and 2) the time at which the fistPDS(PDS1) in DSn becomes available (i.e. PTS (DSn[PDS1])). In otherwords, it should be set the value that satisfies the followingexpression:DTS(DSn[ICS])≦DTS(DSn[ODS1])DTS(DSn[ICS])≦PTS(DSn[PDS1])

Next, the setting of DTS and PTS, for each PDS belonging to DSn, isexplained.

The time at which each PDS belonging to DSn becomes valid in the CLUTunit 9, is from 1) the time at which ICS is loaded to the Compositionbuffer 16 to 2) the decoding start time for the first ODS(DTS(DSn[ODS1]). In view of this, the PTS value of each PDS belonging tothe DSn (i.e. PDs1-PDSlast) should be set to the value that satisfiesthe following relations:DTS(DSn[ICS])≦PTS(DSn[PDS1])PTS(DSn[PDSj])≦PTS(DSn[PSj+1])≦PTS(DSn[PDSlast])PTS(DSn[PDSlast])≦DTS(DSn[ODS1])

Next, the setting of PTS of “END of Display SetSegment” belonging to DSnis explained. The END that belong to DSn shows the ending of the DSn.Therefore it should be the decoding end time of the last ODS(ODSlast) ofthe DSn. This decoding end time is indicated by PTS(PTS(DSn[ODSlast])),and so PTS of End should be set as the value indicated by the followingexpression:PTS(DSn[END])=PTS(DSn[ODSlast])

In view of the relation with ICS belonging to DSn and DSn+1, ICS in DSnis loaded to the Composition buffer 16 prior to the loading time of thefirst ODS (i.e. ODS1). Therefore PTS in END should be after 1) theloading time of ICS belonging to DSn (i.e. DTS(DSn[ICS])), and before 2)the loading time of ICS belonging to DSn+1 (i.e. DTS(DSn+1[ICS])).Accordingly, the PTS in END should satisfy the following relation:DTS(DSn[ICS])≦PTS(DSn[END])≦DTS(DSn+1[ICS])

On the other hand, the loading time of the first ODS (i.e. ODS1) PTS inEND (i.e. PTS(DSn[END])) should be after the loading time of PDSbelonging to DSn. Accordingly, PTS in END should satisfy the followingrelation:PTS(DSn[PDSlast])≦PTS(DSn[END])

Since the ICS, PDS, ODS to which DTS and PTS are set, are incorporatedin AVClip in advance, it is convenient to describe an interactivecontrol for making the reproduction apparatus execute a certainoperation at the time when a frame of a certain moving picture appearsin the screen. In other words, the above arrangement is convenient todescribe an interactive control that is synchronized closely with themoving picture contents. In addition, the ICS, PDS, and ODS aremultiplexed onto the AVClip itself. Therefore in a case where thesections to which a user would like to perform reproduction control arenumerous, such as some hundreds, it is not necessary to store all theIDS, PDS, and ODS, which correspond to all of the sections, in thememory. Because the ICS, PDS, and ODS are to be read from a BD-ROM, thefollowing arrangement is sufficient. That is, ICS, PDS, and ODS, whichcorrespond to the moving picture section to be played back at themoment, stay resident in the memory. After playback for this movingpicture section is finished, the corresponding ICS, PDS, and ODS aredeleted from the memory, and instead, ICS, PDS, ODS corresponding to thesucceeding moving picture section are stored in the memory. Since ICSPDS, and ODS are to be multiplexed onto AVClip, even if the number ofICS, PDS, and ODS becomes some hundreds, the occupancy of the memory canbe restrained to a minimum required level.

As explained above, the present embodiment has 360 pages of ODS so as torealize animation. Therefore when the button material is grouped underthree states, the ODS will be grouped by 120 pages (i.e. into threebutton-state groups). The button-state groups are arranged so that agroup corresponding to earlier appearing state is placed more to thebeginning, compared to a group corresponding to later appearing state.Because of this, at reproduction, a button-state group corresponding toearlier appearing state is loaded to the reproduction apparatusaccordingly earlier, compared to a button-state group corresponding tolater appearing state. According to this, even if decoding of all the360 pages of ODS has not been complete, at least the initial display isready for being performed if only about ⅓-⅔ of the total ODS iscomplete. Since the initial display operation can be commenced at thecompletion of about ⅓-⅔ of the total ODS, even if there are numerous ODSto be read and decoded, the initial display will not be delayed.Therefore, an interactive screen is executed swiftly, even if the screencontains animation to entertain users.

Third Embodiment

The present embodiment relates to a manufacturing method of a BD-ROM.FIG. 79 illustrates a method of manufacturing the PCS explained in thefirst embodiment.

The manufacturing method of a BD-ROM includes: a material producing stepS201 of photographing an image picture, and of recording correspondingaudio, for example; an authoring step S202 of generating an applicationformat; and a press step S203 of completing the BD-ROM by performingpressing/lamination.

Among these steps, the authoring step directed to BD-ROM includes thefollowing step S204-Step S210.

At Step S204, control information, window define information, palettedefine information, and graphics are described. At Step S205, thecontrol information, the window define information, the palette defineinformation, and the graphics are respectively converted into afunctional segment. At Step S206, PTS in PCS is set, according to whenthe picture to be displayed in sync appears. At Step S207, DTS[ODS] andPTS[ODS] are set, according to the value of PTS [PCS]. At Step S208,DTS[PCS], PTS[PDS], DTS[WDS], and PTS[WDS] are set, according to thevalue of DTS[ODS], and at Step S209, the chronological change, inoccupancy, of each buffer of the planer model is expressed as a graph.At Step S210, it is judged whether the chronological change expressed ingraph satisfies the restriction imposed at the player model. If thejudgment of Step S210 results in the affirmative, a graphics stream iscreated at Step S212, and an AVClip is obtained by multiplexing thegraphics stream with a video stream and an audio stream, which have beencreated separately from the graphics stream. Then the AVClip is made tobe in conformity with the format of the BD-ROM, thereby completing anapplication format.

The above explanation is for the manufacturing method of the recordingmedium according to the first embodiment. A manufacturing method of therecording medium according to the second embodiment is shown in FIG. 80.In FIG. 80, Step S304-Step S308 replace Step S204-Step S208 of FIG. 79.

As follows, Step S304-Step S308 are explained. At Step S304, controlinformation, palette define information, and graphics are described. AtStep S305, the control information, the palette define information, andthe graphics are respectively converted into a functional segment. AtStep S306, PTS in ICS is set, according to when the picture to bedisplayed in sync appears. Then at Step S307, DTS[ODS] and PTS[ODS] areset, according to the value of PTS[ICS]. At Step S308, DTS[ICS] andPTS[PDS] are set, according to the value of DTS[ODS].

(Note)

Needless to say, the above explanation does not show all the embodimentsand use form of the present invention. The present invention is alsorealized by an embodiment to which any of the following modifications(A), (B), (C), (D), . . . etc. is added. Please note that the inventionsin the claims of the present invention are broadened or generalizeddescriptions of either the above-described embodiments, or the modifiedembodiments based on the following modifications. The extent of thebroadening and generalization reflects the state of the art at the timeof filing.

(A) In all the embodiments, the recording medium according to thepresent invention is assumed to be a BD-ROM. However, thecharacteristics of the recording medium of the present invention lies onthe graphics stream stored in the recording medium, and thischaracteristics does not rely on the physical nature of a BD-ROM. Inother words, any recording medium that is operable to record a graphicsstream may be used in realizing the present invention. The examplesinclude: an optical disk such as DVD-ROM, DVD-RAM, DVD-RW, DVD-R,DVD+RW, DVD+R, CD-R, CD-RW; and an optical magnetic disk such as PD andMD. The examples further include a semiconductor memory card such as acompact flash card, a smart media, memory stick, multimedia card, and aPCM-CIA card. Still further, the examples include: a magnetic recordingdisk such as a flexible disk, SuperDisk, Clik!; and a removable harddisk drive such as ORB, Jaz, SparQ, SyJet, EZFley, and micro drive. Theexamples also include a hard disk incorporated in an apparatus.

(B) In the all the embodiments, the reproduction apparatus decodes theAVClip stored in the BD-ROM, before outputting it to a television.However, a structure is also possible in which the reproductionapparatus is a mere BD-ROM drive, and the other components are includedin the television. In this case, the reproduction apparatus and thetelevision may be connected to each other via an IEEE1394, to constitutea home network. The reproduction apparatus of the embodiments is for usewith the television connected thereto. However, the reproductionapparatus may be integral with a display. Furthermore, in thereproduction apparatus of each of the embodiments, only the system LSI(integrated circuit), which is the essence of the processing, may beconsidered the invention. The reproduction apparatus and the integratedcircuit are both the inventions described in the present specification,and so the act of manufacturing a reproduction apparatus having any ofthe stated form and manner, based on the internal structure of thereproduction apparatus of the first embodiment is also an embodiment ofthe present invention. Any act of transfer whether incurring charge ornot (sales if incurring charge, and gifts if not incurring charge),rent, and import constitute an embodiment of the present invention. Inaddition, any act of performing these transfer and rent, through theexhibition at the shops, catalogue solicitation, and pamphletdistribution, also constitutes an embodiment of the present invention.

(C) The information processing shown in each flowchart is concretelyrealized using hardware resources. Therefore any programs whoseprocesses are shown in the flowcharts may respectively constitute anindependent invention. All the embodiments relating to the programsassume that a program is in the form incorporated in the correspondingreproduction apparatus. However, The program itself, shown in the firstembodiment, may be an independent embodiment from the correspondingreproduction apparatus. The embodiment of a program itself includes: (1)act of manufacturing the program; (2) act of transfer the programwhether incurring charge or not; (3) act of renting, (4) act ofimporting; (5) act of providing to the public through an interactiveelectronic communication circuit; and (6) offering the transfer togeneral users, through exhibition at shops, catalogue solicitation,pamphlet distribution, and so on.

(D) If the time concept existing in each step, which is executedchronologically in every flowchart, is considered an indispensablefactor for specifying the present invention, then each process in theflowchart is interpreted to disclose a use pattern of the reproductionmethod. If the processes of the above-explained flowcharts are executedby chronologically executing each step therein, so as to be effectivefor and instrumental in achieving the object of the present invention,that will correspond to the embodiment of the recording method of thepresent invention.

(E) When being recorded to the BD-ROM, it is desirable that each TSpacket constituting an AVClip is assigned an extra header. The extraheader is called “TP_extra_header”, includes “Arrival_Time_Stamp” and“copy_permission_indicator”, and has 4-byte length.TP_extra_header-assigned TS packets (EX-assigned TS packet) are dividedinto groups each including 32 TS packets, and are written into threesectors. The total size of the groups each composed of 32 EX assigned TSpackets is 6144 bytes (=32*192), which is equal to the total size of thethree sectors (6144 bytes (=2048*3). 32 EX-assigned TS packets stored inone sector is referred to as “Aligned Unit”.

When used in the home network connected through the IEEE1394, thereproduction apparatus performs Aligned Unit transmission by thefollowing transmission processing. That is, the sender apparatus removesTS_extra_header from each of the 32 EX-assigned TS packets in an AlignedUnit, encodes the TS packets's main bodies, and outputs them. Inoutputting the TS packets, isochronous packets are inserted in manyplaces between the TS packets. The exact insertion places are based onthe time shown by Arrival_Time_Stamp of TS_extra_header. In response tooutputting of the TS packets, the reproduction apparatus outputsDTCP_Descriptor. DTCP_Descriptor signifies the copy permit/prohibitsetting of TP_extra_header. Therefore, if DTCP_Descriptor is describedto signify “prohibit copy”, the TS packets will not be recorded onanother apparatus, at the time when they are used in the home networkconnected through the IEEE1394.

(F) The digital stream in each of the embodiments is an AVClip in theBD-ROM standard. However, it may alternatively be a VOB (Video Object)in the DVD-Video standard or in the DVD-Video Recording standard. A VOBis a program stream conforming to the ISO_IEC13818-1 standard, and isobtained by multiplexing a video stream and an audio stream. The videostream in an AVClip may alternatively be in MPEG4 method or in WMVmethod. Furthermore, an audio stream may alternatively be in Dolby=AC3method, MP3 method, MPEG-AAC method, or dts method.

(G) The movie works in the embodiments may be obtained by encodinganalogue image signals, or stream data made up of transport streamsbroadcasted through a digital broadcast. Furthermore, a content may beobtained by encoding analogue/digital movie signals recorded onvideotape, or may be a digital work distributed from a distributionserver.

(H) The Graphics Object, shown in the first and second embodiments israster data that has been encoded in run-length encoding method. Thereason why the run-length encoding method is used ascompressing/encoding method of Graphics Objects is that the run-lengthencoding method is the most suitable for compressing/decompressingsubtitles. Subtitles are characteristic in that the continual length ofone pixel value in horizontal direction is comparatively long.Therefore, if using the run-length encoding compression, a highcompression rate is obtained. In addition, the load incident todecompression is not much, and so is suitable for creating software fordecoding processing. In the present invention, compression/decompressionmethod used for the subtitles is used for the Graphics Object, so thatone apparatus structure for performing decoding is shared between thesubtitles and the Graphics Objects. However, adopting of run-lengthencoding method for Graphics Objects is not an indispensable feature ofthe present invention, and Graphics Objects may alternatively be PNGdata. In addition, raster data may be vector data, or transparentimages.

(I) Display effects of PCS may be given to graphics of subtitlesselected according to the language setting at the apparatus side. Bythis, the display effect used to be realized by characters representedby the moving picture main body in current DVDs is able to be realizedusing the subtitle graphics displayed according to the language settingat the apparatus side. This is practically very valuable.

(J) Display effects of PCS may be given to graphics of subtitlesselected by the apparatus side according to the display setting.Specifically, graphics for various display modes such as wide vision,pan scan, and letter box have been recorded on a BD-ROM, and theapparatus selects one of them according to the setting of the televisionto which the apparatus is connected to, and displays the selected typeof graphics. In this case, a display effect will be given to thesubtitle graphics displayed in the above way. Therefore the subtitlegraphics will look better. By this, the display effect used to berealized by the characters represented by the moving picture main bodyin the current DVDs is able to be realized using the subtitle graphicsdisplays according to the display setting at the apparatus side. This ispractically very valuable.

(K) In the first embodiment, the write rate Rc to the Graphics Plane isdefined so that the window size is 25% of the entire size, so thatGraphics Plane clearing and re-rendering are possible within one videoframe. However alternatively, if assuming that the vertical retrace timeis 25% of 1/29.93, then the Rc will be 1 Gbps. By setting RC as such,the graphics display will be facilitated. This is practically veryvaluable.

In addition to writing at the vertical retrace time, writingsynchronized with the write scan may be simultaneously performed. Bythis, even if the write rate is Rc=256 Mbps, display will befacilitated.

(L) In each of the embodiments, the reproduction apparatus is equippedwith a Graphics Plane. However, instead of this Graphics Plane, a linebuffer that can store decompressed pixels for one line may be mounted tothe reproduction apparatus. Since the conversion into an image signalcan be performed for each horizontal row (i.e. line), if such a linebuffer is provided, the reproduction apparatus is able to perform theconversion into an image signal.

(M) Subtitles being graphics are explained above, as character sequencesfor representing words uttered in the movie works. However, thesubtitles may contain a combination of figure, character, and color, asconstituting a trademark. Further, the subtitles may contain all sortsof national marks, official marks adopted by nations for supervision andauthorization, marks of international organizations, marks representingplaces of origin of specific commodities, and the like.

(N) The first embodiment assumes that subtitles are displayed at theupper/lower parts of the screen, and so the windows are defined at theupper/lower parts of the Graphics Plane, accordingly. However, it isalso possible to define the windows at the right/left parts of theGraphics Plane. This is useful in displaying Japanese subtitles inlongitudinal direction.

(O) AVClip in each of the embodiments constitutes a movie work. However,AVClip may be for realizing “karaoke” (accompaniment of prerecordedtape). In this case, in the course of a song, PCS may realize a displayeffect that changes the color of subtitles, for example.

(P) In a case where plural reproduction paths joint to one another, andthat the default selected button changes depending on which reproductionpath is to be taken, the following arrangement is preferable. That is,reproduction control in the dynamic scenario is described so that at thetime when each reproduction path is passed, the eigenvalue for thereproduction path is set at the register of the reproduction apparatus,and that the reproduction processes are described so that buttons willbe set in a selected state according to the values set in the register.By this arrangement, the button to be in a selected state can be changedaccording to which reproduction path is to be passed.

INDUSTRIAL APPLICABILITY

A recording medium and a reproduction apparatus, of the presentinvention, realize a subtitle display having display effect and aninteractive display containing animation, and so help provide the marketwith movie works of high added values, which helps invigorate the moviemarket and the consumer goods market. Accordingly, the recording mediumand the reproduction apparatus, of the present invention, are veryuseful in the movie industry and the consumer goods industry.

1. A recording medium storing therein a digital stream that is obtainedby multiplexing a graphics stream and a video stream, the graphicsstream being a packet sequence that includes a data packet storinggraphics data and a control packet storing control information, whereinthe data packet has a time stamp whose value indicates a decoding timeof the graphics data, and the control packet has a time stamp whosevalue indicates a time at which the graphics data, after being decoded,is displayed combined with the video stream.
 2. The recording medium ofclaim 1, wherein the control information includes type information thatindicates a memory management start, the time stamp of the controlpacket is a presentation time stamp, and the control packet furtherincludes a decode time stamp whose value indicates a point of areproduction timeline of the digital stream, which corresponds to thememory management start, and a time at which the control information isread to a memory.
 3. The recording medium of claim 2, wherein the valueof the presentation time stamp is obtained by adding a predeterminedvalue to the value of the decode time stamp, where the predeterminedvalue is based on: a longer one of a period required for clearing of ascreen, and a period required for decoding of the graphics data; and aperiod required for writing of the graphics data to the screen.
 4. Therecording medium of claim 1, wherein the control information is one of:information indicating a difference with preceding control information;a duplicate of the preceding control information; and informationindicating that memory management continues from another digital stream,the time stamp of the control packet is a presentation time stamp, andthe control packet further includes a decode time stamp whose valueindicates a time at which the control information is read to a memory ofa reproduction apparatus.
 5. The recording medium of claim 4, whereinthe value of the presentation time stamp is obtained by adding apredetermined value to the value of the decode time stamp, where thepredetermined value is based on: a longer one of a period required forclearing of a screen and a period required for decoding of the graphicsdata; and a period required for writing of the graphics data to thescreen.
 6. The recording medium of claim 5, wherein the screen clearingis clearing of a window area of the graphics plane, the graphics planebeing at which the graphics data is to be disposed, the graphics streamincludes a window define packet storing window information, and thewindow information indicates the window area of the graphics plane. 7.The recording medium of claim 6, wherein a value of a decode time stampof the window define packet indicates a time in a range between a timeshown by the decode time stamp of the control packet and a time shown bythe time stamp of the data packet, inclusive, and a value of apresentation time stamp of the window define packet indicates a timerecommended to start transferring the graphics data for display.
 8. Therecording medium of claim 7, wherein. the graphics stream includes apalette define packet storing palette information that defines colorsetting used in display of the graphics data, and a value of apresentation time stamp of the palette define packet indicates a timewhen the palette information becomes available for use.
 9. The recordingmedium of claim 1, wherein the graphics stream has two sets of graphicsdata, and a value of the time stamp of the control packet is obtained byadding a predetermined value to a value of the time stamp of the datapacket, where the predetermined value is a total of a period x and aperiod y, the period x being a longer one of: a total of a periodrequired for clearing a screen and a period required for writing theformer of the two sets of graphics data after decoded, to the screen;and a total period required for decoding the two sets of graphics data,and the period y being a period required for writing, to a screen, thelatter of the two sets of graphics data after decoded.
 10. The recordingmedium of claim 1, wherein the graphics stream includes a plurality ofdata packets storing graphics data, including the data packet, the datapackets make up a data-packet sequence, the data-packet sequenceconstitutes graphical button materials, the time indicated by the timestamp of the control packet is an initial display time of an interactivescreen, and a time at which the initial display is performed is obtainedby adding a predetermined period to a decoding ending time of graphicsdata positioned at a middle of the data-packet sequence, where thepredetermined period is based on a total of a first period required forclearing a graphics plane of a reproduction apparatus and a secondperiod required to perform writing for the initial display, to thegraphics plane.
 11. The recording medium of claim 10, wherein if buttonmaterials, which are to be in a selected state as a default in theinitial display, are statically determined, the control informationincludes default selected information specifying the button materials,and the second period is a period required f or writing, to the graphicsplane, a total of firstly displayed part of decompressed graphics datarepresenting a selected state of one button; and firstly displayed partof decompressed graphics data representing normal states of buttonsdifferent from the button.
 12. The recording medium of claim 10, whereinif button materials, which are to be in a selected state as a default inthe initial display, dynamically change, the second period is a periodrequired for writing data of size T, to the graphics plane, where thesize T is obtained by adding together sizes x of all buttons, a size xbeing obtained by an expression of:size x=max(N−first,S−first), N-first representing firstly displayed partof entire graphics data representing a normal state, S-firstrepresenting firstly displayed part of entire graphics data representinga selected state, and max(a,b) being a formula for indicating one of “a”and “b” that has a larger size therebetween.
 13. The recording medium ofclaim 10, wherein the control information includes type information, andwhen corresponding type information indicates a memory management start,the predetermined period is a total of the first period and the secondperiod, and when the type information indicates either a difference withpreceding control information, or a duplicate of the preceding controlinformation, the predetermined period is the second period.
 14. Therecording medium of claim 10, wherein the graphics stream includes apalette define packet storing palette information that defines colorsetting used in display of the graphics data in the plurality of datapackets, and a value of a presentation time stamp of the palette definepacket indicates a time when the palette information becomes availablefor use.
 15. A reproduction apparatus for a digital stream that isobtained by multiplexing a video stream and a graphics stream, thegraphics stream being a packet sequence that includes a data packetstoring graphics data and a control packet storing control information,the reproduction apparatus comprising: a video decoder operable toobtain a moving picture by decoding the video stream; and a graphicsdecoder including: a processor that starts decoding of graphics datawhen a current reproduction point of the video stream reaches a pointshown by a decode time stamp of the data packet, and ends the decodingwhen the current reproduction point reaches a point shown by apresentation time stamp of the data packet; and a controller that,according to a presentation time stamp of the control packet, combines,with a video stream, the graphics data after the decoding, and displaysthe graphics data combined with the video stream.
 16. The reproductionapparatus of claim 15, wherein the graphics decoder includes: a codeddata buffer for storing compressed graphics data, and an object bufferfor storing decompressed graphics data resulting from the decoding ofthe processor, and the reproduction apparatus includes a graphics planefor storing part of the decompressed graphics data obtained by thedecoding of the processor, the part of the decompressed graphics databeing used for display, the controller judges whether type informationincluded in the control information indicates a memory management start,and if judging affirmatively, clears the coded data buffer, the objectbuffer, and the graphics plane when the current reproduction point ofthe video stream reaches a point shown by a decode time stamp of thecontrol packet.
 17. The reproduction apparatus of claim 16, wherein thecontroller, when the current reproduction point reaches a point shown bythe decode time stamp of the control packet, starts clearing thegraphics plane, and the processor, when the current reproduction pointreaches a point shown by the decode time stamp of the data packet,decodes the graphics data in parallel with the clearing of thecontroller, and the controller, when the current reproduction pointreaches a point shown by the presentation time stamp of the controlpacket, writes the decompressed graphics data, to the graphics plane.18. The reproduction apparatus of claim 17, wherein the value of thepresentation time stamp of the control packet is obtained by adding apredetermined value to a value of the decode time stamp of the controlpacket, where the predetermined value is based on: a longer one of aperiod required for clearing of a screen, and a period required fordecoding of the graphics data; and a period required for writing of thegraphics data to the screen, and the controller, when the currentreproduction point reaches a point shown by the presentation time stampof the control packet, displays content of the graphics plane.
 19. Thereproduction apparatus of claim 16, wherein the graphics stream includesa window define packet storing window information, the controller clearsthe graphics plane when the type information indicates a memorymanagement start, and clears a window area of the graphics plane, whichis indicated by the window information, when the type informationindicates other than a memory management start.
 20. The reproductionapparatus of claim 19, wherein control information, whose typeinformation does not indicate a memory management start, is one of:information indicating a difference with preceding control information;a duplicate of the preceding control information; and informationindicating that memory management continues from another digital stream,and when the current reproduction point reaches a point shown by adecode time stamp of a control packet including said control informationwhose type information does not indicate a memory management start, thecontroller reads corresponding control information to a memory of thereproduction apparatus.
 21. The reproduction apparatus of claim 19,wherein a decode time stamp of the window define packet indicates a timein a range between a time shown by a decode time stamp of the controlpacket and a time shown by a decode time stamp of the data packet,inclusive, and decompressed graphics is written to a graphics plane, byreferring to a value of a presentation time stamp of the window definepacket.
 22. The reproduction apparatus of claim 21, wherein the graphicsstream includes a palette define packet storing palette information thatdefines color setting used in display of the graphics data, thereproduction apparatus includes a color conversion unit operable toperform color conversion on graphics stored in the graphics plane, andthe controller, when the current reproduction point reaches a pointshown by a presentation time stamp of the palette define packet, setsthe palette information in the color conversion unit.
 23. Thereproduction apparatus of claim 19, wherein the graphics stream has twosets of graphics data, a value of a presentation time stamp of thecontrol packet is obtained by adding a predetermined value to a value ofa presentation time stamp of the data packet, where the predeterminedvalue is a total of a period x and a period y, the period x being alonger one of: a total of a period required for clearing a screen and aperiod required for writing the former of the two sets of graphics dataafter decoded, to the screen; and a total period required for decodingthe two sets of graphics data, and the period y being a period requiredfor writing, to a screen, the latter of the two sets of graphics dataafter decoded, and the controller, when the current reproduction pointreaches a point shown by a presentation time stamp of the controlpacket, displays content of the graphics plane.
 24. The reproductionapparatus of claim 15, wherein the graphics stream includes a pluralityof data packets storing graphics data, including the data packet, thedata packets make up a data-packet sequence, the data-packet sequenceconstitutes graphical button materials, the graphics decoder combines,with a moving picture, an interactive screen that includes the graphicalbutton materials, and displays the interactive screen combined with themoving picture, and the reproduction apparatus performs, when thereproduction point of the video stream reaches a point shown by a timestamp of the control packet, an initial display of the interactivescreen by using graphics data positioned up to a middle of thedata-packet sequence, and the reproduction apparatus performs, when auser operation is received, updating of the interactive screen by usinggraphics data positioned after the middle of the data-packet sequence.25. The reproduction apparatus of claim 24, wherein a time at which theinitial display of the interactive screen is performed is obtained byadding a predetermined period to a decoding ending time of graphics datapositioned at the middle of the first-packet sequence, where thepredetermined period is based on a total of a first period required forclearing a graphics plane of the reproduction apparatus and a secondperiod required to perform writing for the initial display, to thegraphics plane.
 26. The reproduction apparatus of claim 25, wherein iftype information included in the control information indicates a memorymanagement start, the controller sets, as the predetermined period, atotal of the first period and the second period, and if the typeinformation indicates either a difference with preceding controlinformation, or a duplicate of the preceding control information, thecontroller sets the second period as the predetermined period.
 27. Thereproduction apparatus of claim 24, wherein if button materials, whichare to be in a selected state as a default in the initial display, arestatically determined, the control information includes default selectedinformation specifying the button materials, and calculates the secondperiod, based on a total of firstly displayed part of decompressedgraphics data representing a selected state of one button and firstlydisplayed part of decompressed graphics data representing normal statesof buttons different from the button.
 28. The reproduction apparatus ofclaim 24, wherein if button materials, which are to be in a selectedstate as a default in the initial display, dynamically change, thesecond period is a period required for writing data of size T, to thegraphics plane, where the size T is obtained by adding together sizes xof all buttons, a size x being obtained by an expression of:size x=max(N−first,S−first), N-first representing firstly displayed partof entire graphics data representing a normal state, S-firstrepresenting firstly displayed part of entire graphics data representinga selected state, and max(a,b) being a formula for indicating one of “a”and “b” that has a larger size therebetween.
 29. A recording method fora recording medium, the recording method comprising: a step of creatingapplication data; and a step of recording the created application datato the recording medium, wherein the application data includes a digitalstream that is obtained by multiplexing a graphics stream and a videostream, the graphics stream being a packet sequence that includes a datapacket storing graphics data and a control packet storing controlinformation, the data packet has a time stamp whose value indicates adecoding time of the graphics data, and the control packet has a timestamp whose value indicates a time at which the graphics data, afterbeing decoded, is displayed combined with the video stream.
 30. Anintegrated circuit used in a reproduction apparatus for a digitalstream, the digital stream being obtained by multiplexing a video streamand a graphics stream, the graphics stream being a packet sequence thatincludes a data packet storing graphics data and a control packetstoring control information, the integrated circuit comprising: a videodecoder operable to obtain a moving picture by decoding the videostream; and a graphics decoder including: a processor that startsdecoding of graphics data when a current reproduction point of the videostream reaches a point shown by a decode time stamp of the data packet,and ends the decoding when the current reproduction point reaches apoint shown by a presentation time stamp of the data packet; and acontroller that, according to a presentation time stamp of the controlpacket, displays the graphics data, after being decoded, combined withthe video stream.
 31. A program that makes a computer performreproduction of a digital stream, the digital stream being obtained bymultiplexing a video stream and a graphics stream, the graphics streambeing a packet sequence that includes a data packet storing graphicsdata and a control packet storing control information, the programcomprising: a code that makes the computer obtain a moving picture bydecoding the video stream; and a code that makes the computer performgraphics rendering by which the graphics data combined with the movingpicture is displayed, the graphics rendering performed by the computerincluding: a process of starting decoding of graphics data when acurrent reproduction point of the video stream reaches a point shown bya decode time stamp of the data packet, and ending the decoding when thecurrent reproduction point reaches a point shown by a presentation timestamp of the data packet; and a process of, according to a presentationtime stamp of the control packet, displays the graphics data, afterbeing decoded, combined with the video stream.
 32. A reproduction methodfor a digital stream that is obtained by multiplexing a video stream anda graphics stream, the reproduction method comprising: a process ofobtaining a moving picture by decoding the video stream; and a processof performing graphics rendering by which the graphics data combinedwith the moving picture is displayed, the graphics rendering including:a process of starting decoding of graphics data when a currentreproduction point of the video stream reaches a point shown by a decodetime stamp of the data packet, and ending the decoding when the currentreproduction point reaches a point shown by a presentation time stamp ofthe data packet; and a process of, according to a presentation timestamp of the control packet, displays the graphics data, after beingdecoded, combined with the video stream.